This chapter describes the methods by which the committee evaluated the evidence regarding the efficacy and effectiveness of cognitive rehabilitation therapy (CRT) for traumatic brain injury (TBI), including the means by which the committee searched for and organized the literature. The chapter also includes an assessment of the quality of study design and its related impact on how the studies were evaluated. The committee searched for and reviewed evidence of CRT interventions by either specific cognitive domain (i.e., memory, attention, executive function, visuospatial perception, and communication and language) or multi-modal/comprehensive CRT.
The committee iteratively developed a protocol to address the following questions:
• Do cognitive rehabilitation interventions improve function and reduce cognitive deficits in adults with mild or moderate-severe TBI?
• Are any cognitive rehabilitation interventions associated with risk for adverse events or harm?
• Are cognitive rehabilitation interventions delivered through telehealth technology safe and efficacious?
The committee reviewed published systematic reviews (Cicerone et al. 2000, 2005, 2011; ECRI 2009; Kennedy et al. 2008) and worked with a research librarian to develop search strategies to identify pertinent evidence. The strategy included searches in the following electronic bibliographic
databases: Medline, EMBase, PsycInfo, Education Resources Information Center (ERIC), and Cochrane (e.g., Cochrane DB of Systematic Reviews, Database of Reviews of Effects [DARE] and Cochrane Central Register of Controlled Trials). Key terms and Medical Subject Headings (keywords for Medline) focused on subject areas related to brain injury and CRT. Strategy parameters limited searches to human subjects, the English language, and results published between January 1991 and April 2011. The time period was chosen to include articles prior to Operation Desert Storm, which began in 1991. Setting time parameters allowed for the evaluation of the most recent research of relevance, acknowledging that more recent studies build on the evidence base created by older literature. Furthermore, because TBI has occurred more frequently among service members in recent conflicts, beginning with Operation Desert Storm, research in the field of TBI and CRT has greatly expanded since that time. To ensure it captured all relevant studies, the committee conducted a secondary search to identify articles not found during the electronic search. This practice is common when conducting a literature review. To complete the secondary search, the committee extensively examined the bibliographies of previously published systematic reviews on cognitive rehabilitation therapy for TBI, reading all full-text articles contained in those reference lists that had not been identified in the primary search. The committee determined it would include studies from these reference lists that met inclusion criteria (as described in Box 6-1), regardless of publishing date. The committee reviewed many excellent studies during this process; however, not all studies met inclusion criteria. The secondary search identified 12 additional articles, 2 of which were published prior to 1991. No other study published prior to 1991, that the committee reviewed, met inclusion criteria.
The committee focused on studies that used one or several forms of CRT to ameliorate the effects of traumatic brain injury. Per its charge, the committee considered CRT for TBI across all severities of injury (mild and moderate-severe) and across all stages of recovery (acute, subacute, and chronic). For the purposes of this review, the committee defined the time periods for acute, subacute, and chronic phases of recovery following TBI (see Table 6-1). The searches limited the scope of terms to traumatic brain injury, and did not consider other forms of acquired brain injury, such as those due to stroke, ischemia, infection, or malignancy. Similarly, the committee did not review literature on the effects of CRT for non-TBI cognitive conditions, such as schizophrenia, dementia, or learning disabilities.
The initial electronic search identified 856 studies. Upon review of titles and abstracts, 121 studies were selected for more detailed review. At least two committee members reviewed each full text article to determine relevancy, based on the committee’s inclusion and exclusion criteria, shown in Box 6-1. Upon full-text review, 43 studies were excluded. An additional
1.1 Sample is composed of individuals with TBI (open or closed, with or without secondary hypoxic/ischemic injury), as evidenced by
a. Initial loss/alteration of consciousness on clinical assessment (abnormal GCS or posttraumatic amnesia); OR
b. Findings on neuro-imaging consistent with TBI; OR
c. Focal impairment on neurologic exam consistent with TBI; OR
d. Documentation of injury for patients with mild TBI (plausible history is sufficient for patients with moderate-severe TBI);
1.2 Sample is mixed between TBI and non-TBI but results are reported separately for TBI subjects (who meet the above definition); OR
1.3 Sample is mixed but contains a majority of TBI participants; AND
1.4 Sample is composed of individuals age 18 or older.
2.1 The intervention is sufficiently described for classification/categorization as CRT; AND
2.2 Studies that primarily evaluated drug efficacy are excluded.
3.0 Outcome Measures
3.1 Outcome measure(s) could be either objective or subjective measures; AND
3.2 Studies where the only outcome measures are performance of tasks that were directly practiced in the treatment protocol are excluded.
4.0 Study Design
4.1 Uncontrolled case reports or case series are excluded.
4.2 Single subject experimental designs (i.e., designs focusing on outcome within a subject, while incorporating experimental controls) are included.
4.3 For pre-post studies conducted during a postinjury period and over a duration in which substantial change might be expected in the primary outcome(s), studies with no comparison group (since measured improvement may be “spontaneous”) (e.g., if mild TBI occurred over 6 months or fewer, and moderate-severe TBI occurred over 12 months or fewer) are excluded.
4.4 For studies conducted in a postacute period, pre-post studies with no comparison group and only subjective self-report outcomes (which may be strongly affected by expectation) are excluded.
5.1 Only studies available in the English language are included.
|Mild TBI||Moderate-Severe TBI|
|Acute||< 3 months||Acute hospital care|
|Subacute||> 3 months < 6 months||Inpatient rehabilitation|
|Chronic||> 6 months < 12 months||Outpatient rehabilitation|
12 studies were added through the secondary search (i.e., culling reference lists), for a total of 90 studies upon which the committee based its conclusions.
The committee designed forms for extracting and summarizing data from each study, including information about study design and methods, patient characteristics, treatment interventions and outcomes (i.e., World Health Organization International Classification of Functioning, Disability and Health [WHO-ICF] framework), and funding source. The Institute of Medicine (IOM) contracted two individuals with knowledge and expertise in CRT to extract data from selected studies; these individuals (i.e., coders) were neither IOM staff nor members of the committee. At least two committee members read each of the original articles and compared information from the studies to the evidence tables completed by the independent coders. The committee assessed methodologic limitations of studies, described each study, and synthesized the evidence in a narrative form.
The committee conceptually categorized CRT interventions as either (1) modular strategies aimed at attention, memory, executive function, language or social communication, or visuospatial deficits or (2) multi-modal, comprehensive strategies. The intent of the therapy was categorized as restorative or compensatory and the goals and setting of therapy as decontextualized or contextualized. Compensatory strategies for cognitive impairment (e.g., memory aids) that involved changes to the environment were categorized as external; strategies that did not involve environmental changes were categorized as internal. The committee recognizes that conceptual categorizations may not translate to real-world application; these categories were useful for organizing and evaluating of the evidence. The separation between modular and multi-modal/comprehensive strategies was specific to the committee’s charge.
The committee found 90 studies that met selection criteria. Of these, 37 were randomized controlled trials (RCTs) (2 of the 37 addressed both memory and attention deficits); 15 were nonrandomized, parallel group
controlled trials; 19 were pre-post single group studies; and 15 were reports of one or more single subject, multiple baseline experiments. Of the studies, 21 addressed multi-modal or comprehensive cognitive rehabilitation, including RCTs, crossover group, nonrandomized controlled parallel group, and pre-post single group designs. Table 6-2 provides information about the number of studies, by design, were identified in each cognitive domain or multi-modal/comprehensive CRT.
The committee did not identify any CRT studies in the acute phase of recovery following TBI. Several studies of multimodal/comprehensive treatment programs were conducted in the subacute phase, but most of the modular treatment studies were conducted in the chronic phase. Few studies included in this review specifically enrolled individuals with mild TBI, or reported results separately for those with mild injuries who were enrolled in mixed studies. Where evidence exists with respect to treatment of participants in the subacute phase, or those with mild injuries, the committee highlighted these studies and relevant findings.
As charged, the committee reviewed evidence across intervention types to determine if there was evidence regarding efficacy or effectiveness in individual cognitive domains and multi-modal/comprehensive CRT. Studies were assessed for improvements in objective measures of benefit, or short-and long-term treatment effects. Studies were also assessed for subjective self-reports by patients or family members of treatment benefit, or patient-centered outcomes. These distinctions are useful because achievements on objective measures of benefit may not translate into improvement in real-world functioning. It is important to note that standards for other aspects of medical practice and research, such as pharmacologic agents, do not require patient-centered outcomes, such as return to work or improved quality of life, to show any treatment benefit or to receive regulatory
|Study Design by Treatment Domain or Strategy||Attention||Executive Function||Language and Social Communication||Memory||Multimodal/ Comprehensive CRT|
|Nonrandomized, Parallel Group||0||4||1||2||8|
|Pre-Post Single Group||2||4||0||6||7|
|Single Subject Multiple Baseline||1||8||0||6||0|
approval or coverage by insurers. Therefore, the absence of patient-centered outcomes did not necessarily detract from a study’s evidence base. However, the committee acknowledges that these are important outcomes to report, especially in goal-oriented and interactive rehabilitation. The committee also reviewed studies where use of telehealth technology was employed, to determine the safety and efficacy of CRT applied through these technologies, compared to interventions applied in clinical settings. The potential for adverse effects or harm was also evaluated among the included studies.
Also per its charge, the committee separately evaluated studies by the type of comparator arm, including inert or no treatment, a non-CRT treatment, or another form of CRT. Varying comparators were not considered more or less useful, only that they answer different questions about the value of CRT for TBI. To determine efficacy, the committee relied on studies that compared the primary CRT treatment to either no treatment or a non-CRT treatment. To determine effectiveness, the committee evaluated studies comparing CRT treatment to another form of CRT. Comparative effectiveness studies may be premature without preceding efficacy trials of the interventions applied in each arm. Furthermore, cognitive processes are complex and intertwined. Likewise, treatment activities generally employ multifaceted tasks. Therefore, attempts to predict a highly specific effect of one CRT intervention (e.g., attention process training) on an isolated cognitive process (e.g., attention) is difficult without considering the effect another CRT treatment (e.g., notebook training for a memory deficit) may have on the original cognitive function of interest (e.g., attention). For these reasons, interventions comparing one form of CRT to another were less helpful in determining the impact of a specific intervention to improve a specific cognitive function.
The committee discussed at length the need to establish relevant criteria for interpreting the studies under review to address the study questions asked by the Department of Defense. The committee reached consensus on the grading system shown in Box 6-2.
In an interactive and collaborative process, the committee graded the overall body of evidence for each CRT category (by domain, TBI severity, and recovery phase [for example, CRT interventions for attention in moderate-severe TBI patients in the chronic phase of recovery]). To draw conclusions about treatment efficacy or effectiveness, the committee qualitatively assessed the strength of individual studies, as well as the consistency of treatment effect among studies. The strength of each study was based on an iterative quality assessment, considering study design, size of the sample, reported characteristics of the sample (e.g., injury severity) and treatment (e.g., dosage, frequency, and timing), control for potentially confounding factors, magnitude of the treatment effect, statistical significance of the findings, and the length of follow-up.
• None or Not informative (0): No evidence because the intervention has not been studied or uninformative evidence because of null results from flawed or otherwise limited studies
• Limited (+): Interpretable result from a single study or mixed results from two or more studies
• Modest (++): Two or more studies reporting interpretable, informative, and largely similar results
• Strong (+++): Reproducible, consistent, and decisive findings from two or more independent studies characterized by the following: (1) replication, reflected by the number of studies (multiple, at least two) in the same direction (2) statistical power and scope of studies (N size of the study and single or multi-site); and (3) quality of the study design to measure appropriate end-points (to evaluate efficacy and safety) and minimize bias and confounding
The committee gave more weight to controlled designs than uncontrolled (e.g., results of RCTs were given more weight than results from pre-post single group designs). Conclusions were not based solely on findings from uncontrolled studies; however, the committee included pre-post single group designs and single subject, multiple baseline experiments in the review because uncontrolled studies may include useful information about nascent interventions or lend support to a controlled design with similar results. Where evidence was informative, the committee specifically identifies the treatment mode and cites the one or more studies that led to its conclusion.
In making its conclusions, the committee found most informative those studies that failed the fewest criteria. Evidence ruled “limited” does not mean an intervention was inadequate; it may simply mean there were methodological flaws in the study design. As is commonly seen among studies evaluating rehabilitation strategies, the overall limitations of the evidence were due to a number of identified issues in study designs. Some of these issues involved the heterogeneity and lack of operational definitions of different forms of CRT; small sample sizes; the variety of premorbid, comorbid, and environmental factors that may moderate the value of a given form of CRT across patients; and the range of outcomes that may be targeted.
None of the included studies were absent of limitations in study design. About one-third of the RCTs were small studies involving fewer than 20 participants, and about 20 percent were larger studies involving more than 50 participants. The severity of TBI was described as moderate or severe in 22 trials and as mild to moderate or mild to moderate-severe in 5 trials, and was unclearly specified in 10 trials. Most trials included participants who were many months postinjury (i.e., chronic TBI). Settings for 7 of the larger trials included a suburban rehabilitation hospital in the northeastern United States (Cicerone et al. 2008), a rehabilitation center in Colorado (Dahlberg et al. 2007), three brain injury units in Sydney, Australia (McDonald et al. 2008), a neuropsychological rehabilitation program at a metropolitan medical center in New York (Rath et al. 2003), a U.S. military medical referral center (Salazar et al. 2000),1 four U.S. Department of Veterans Affairs’ acute inpatient rehabilitation programs (Vanderploeg et al. 2008), and an academic neurosurgical unit in Hong Kong (Zhu et al. 2007). About 20 percent of the trials described adequate methods to generate random allocation sequences and assure allocation concealment. A few trials used quasi-experimental designs that matched patient characteristics such as age and severity of injury before or after randomization. Few reports detailed a priori sample size calculations. Some trial reports provided consort figures or detailed descriptions of follow-up including number of participants randomized to groups, completeness of follow-up, and amount of missing data by group; most trials did not report all of this information. Few trial reports detailed analytic methods that were used to handle missing data or specified numbers of people included in analyses of each outcome measure that was reported. Trials generally evaluated a heterogeneous group of interventions including focused interventions targeted at specific and sometimes narrow deficits and more complex interventions targeted toward multiple deficits. Trials also had heterogeneous comparison groups. Whether participants received co-interventions or ancillary treatments such as antidepressants or pain medications that might augment or interfere with cognitive rehabilitation effects was rarely described. In only a few trials were attempts made to blind personnel administering objective outcome measures to group assignments of trial participants. The limitations of the evidence do not rule out meaningful benefit. The committee did not identify methodological issues in this report to hold CRT research to a higher standard than rehabilitation research at large; it serves merely as an overt discussion of the issues that cloud determination of efficacy and effectiveness.
1 The committee reviewed Salazar et al. 2000, with Braverman et al. 1999, and Warden et al. 2000.
In the chapters that follow, the committee applies the methods and background knowledge described in the present and previous chapters to assess the available evidence on CRT treatments for TBI-related deficits in attention, executive function, language and social communication, memory, and multi-modal/comprehensive CRT (Chapters 7 through 11, respectively). The committee did not identify any relevant literature for treatment of visuospatial perception deficits, which are more common after stroke than TBI. These five chapters include evidence tables with key information about included studies. Chapter 12 summarizes studies that applied telehealth technology, and Chapter 13 describes possible adverse events or harm from CRT. Conclusions are made within each chapter. Conclusions about the evidence were not compared to the findings of other systematic reviews, which the committee deemed beyond its charge.
Each chapter begins with an overview describing the presentation of studies. As various domains required differential distinctions for proper analysis, the chapters do not follow a consistent format. The evidence is organized by the conceptual categories that provided the most use in drawing overall conclusions, dictated by the available body of evidence. The committee did not interpret the evidence differently within these categories. For example, memory strategies were divided by internal, external, or restorative within mild or moderate-severe TBI. Whereas attention strategies were divided by those found in the subacute or chronic phase of recovery in patients with moderate-severe TBI (as no studies were identified of patients with mild TBI with attentional deficits). When the committee found evidence showing treatment benefit, the conclusions explicitly identify the specific intervention and cite the study in which it was described.
Braverman, S. E., J. Spector, D. L. Warden, B. C. Wilson, T. E. Ellis, M. J. Bamdad, and A. M. Salazar. 1999. A multidisciplinary TBI inpatient rehabilitation programme for active duty service members as part of a randomized clinical trial. Brain Injury 13(6):405–415.
Cicerone, K. D., C. Dahlberg, K. Kalmar, D. M. Langenbahn, J. F. Malec, T. F. Bergquist, T. Felicetti, J. T. Giacino, J. P. Harley, D. E. Harrington, J. Herzog, S. Kneipp, L. Laatsch, and P. A. Morse. 2000. Evidence-based cognitive rehabilitation: Recommendations for clinical practice. Archives of Physical Medicine and Rehabilitation 81(12):1596–1615.
Cicerone, K. D., C. Dahlberg, J. F. Malec, D. M. Langenbahn, T. Felicetti, S. Kneipp, W. Ellmo, K. Kalmar, J. T. Giacino, J. P. Harley, L. Laatsch, P. A. Morse, and J. Catanese. 2005. Evidence-based cognitive rehabilitation: Updated review of the literature from 1998 through 2002. Archives of Physical Medicine and Rehabilitation 86(8):1681–1692.
Cicerone, K. D., T. Mott, J. Azulay, M. A. Sharlow-Galella, W. J. Ellmo, S. Paradise, and J. C. Friel. 2008. A randomized controlled trial of holistic neuropsychologic rehabilitation after traumatic brain injury. Archives of Physical Medicine and Rehabilitation 89(12):2239–2249.
Cicerone, K. D., D. M. Langenbahn, C. Braden, J. F. Malec, K. Kalmar, M. Fraas, T. Felicetti, L. Laatsch, J. P. Harley, T. Bergquist, J. Azulay, J. Cantor, and T. Ashman. 2011. Evidence-based cognitive rehabilitation: Updated review of the literature from 2003 through 2008. Archives of Physical Medicine and Rehabilitation 92(4):519–530.
Dahlberg, C. A., C. P. Cusick, L. A. Hawley, J. K. Newman, C. E. Morey, C. L. Harrison-Felix, and G. G. Whiteneck. 2007. Treatment efficacy of social communication skills training after traumatic brain injury: A randomized treatment and deferred treatment controlled trial. Archives of Physical Medicine and Rehabilitation 88(12):1561–1573.
ECRI. 2009. Cognitive Rehabilitation for the Treatment of Traumatic Brain Injury. Plymouth Meeting, PA: ECRI Institute.
Kennedy, M. R., C. Coelho, L. Turkstra, M. Ylvisaker, M. Moore Sohlberg, K. Yorkston, H. H. Chiou, and P. F. Kan. 2008. Intervention for executive functions after traumatic brain injury: A systematic review, meta-analysis and clinical recommendations. Neuro-psychological Rehabilitation 18(3):257–299.
McDonald, S., R. Tate, L. Togher, C. Bornhofen, E. Long, P. Gertler, and R. Bowen. 2008. Social skills treatment for people with severe, chronic acquired brain injuries: A multicenter trial. Archives of Physical Medicine and Rehabilitation 89(9):1648–1659.
Rath, J. F., D. Simon, D. M. Langenbahn, R. L. Sherr, and L. Diller. 2003. Group treatment of problem-solving deficits in outpatients with traumatic brain injury: A randomised outcome study. Neuropsychological Rehabilitation 13(4):461–488.
Salazar, A. M., D. L. Warden, K. Schwab, J. Spector, S. Braverman, J. Walter, R. Cole, M. M. Rosner, E. M. Martin, J. Ecklund, and R. G. Ellenbogen. 2000. Cognitive rehabilitation for traumatic brain injury: A randomized trial. Journal of the American Medical Association 283(23):3075–3081.
Vanderploeg, R. D., K. Schwab, W. C. Walker, J. A. Fraser, B. J. Sigford, E. S. Date, S. G. Scott, G. Curtiss, A. M. Salazar, and D. L. Warden. 2008. Rehabilitation of traumatic brain injury in active duty military personnel and veterans: Defense and Veterans Brain Injury Center randomized controlled trial of two rehabilitation approaches. Archives of Physical Medicine and Rehabilitation 89(12):2227–2238.
Warden, D. L., A. M. Salazar, E. M. Martin, K. A. Schwab, M. Coyle, and J. Walter. 2000. A home program of rehabilitation for moderately severe traumatic brain injury patients. Journal of Head Trauma Rehabilitation 15(5):1092–1102.
Zhu, X. L., W. S. Poon, C. C. H. Chan, and S. S. H. Chan. 2007. Does intensive rehabilitation improve the functional outcome of patients with traumatic brain injury (TBI)? A randomized controlled trial. Brain Injury 21(7):681–690.