National Academies Press: OpenBook
« Previous: 3 The Translational Pipeline and Classification Schemes for CRT Interventions
Suggested Citation:"4 Key Themes." Institute of Medicine. 2013. Cognitive Rehabilitation Therapy for Traumatic Brain Injury: Model Study Protocols and Frameworks to Advance the State of the Science: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18257.
×

4

Key Themes

After the three opening presentations set the stage for discussion, the workshop participants were divided into small groups and asked to analyze three published research papers on cognitive rehabilitation therapy (CRT) for traumatic brain injury (TBI). Workshop participants had been provided with the three papers and a list of questions for each prior to the workshop. Committee members had drafted the discussion questions, and a committee member moderated each small-group discussion. The discussion of the three papers was not intended to reassess the papers’ conclusions; rather, the discussions were designed to elicit participants’ suggestions about the elements of optimal research design and the succession of studies through the maturational process. Participants evaluated variables such as sample size, patient selection, description of interventions, and selection and measurement of outcomes, with an eye toward generalizing their assessments to inform the design of future studies. Following each small-group discussion, a member of each group reported back to the entire group and a free-ranging discussion ensued. This chapter summarizes the key themes in the report-back sessions. As was noted in Chapter 1, all ideas and suggestions expressed by participants do not reflect a consensus and should not be interpreted as such.

Suggested Citation:"4 Key Themes." Institute of Medicine. 2013. Cognitive Rehabilitation Therapy for Traumatic Brain Injury: Model Study Protocols and Frameworks to Advance the State of the Science: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18257.
×

THE TRANSLATIONAL PIPELINE AND RESEARCH DESIGN

Balancing Urgency vs. Evidence

Health service administrators and service providers must choose which interventions to pursue by balancing competing needs: the magnitude of patients’ needs, the magnitude of the intervention’s benefit, the cost of the research, and the cost of the intervention itself. There was a good deal of discussion about the continuum from “implement something immediately” to “wait and research interventions thoroughly.” Participants explored the question, how clear does the evidence need to be for an intervention to be worth implementing? At one end of the continuum is the decision to implement interventions right away (without as strong of evidence of efficacy as might be desired) and to plan subsequent research based on what the implementation showed. At the other end of the continuum is the decision to more rigorously determine which interventions are effective and why, followed then by implementation. Individual participants expressed favor for each strategy, depending on the balance of urgency, the effect size, cost-effectiveness, and other factors.

Promoting Continuity in the Research Pipeline

Several participants agreed on the need for greater continuity in the translational pipeline for using CRT to treat TBI. For any given domain (attention, executive function, language and social communication, visuospatial perception, and memory) and patient population (severity of injury, stage of recovery), there are few published papers that describe in detail what the intervention was, what kind of feedback was measured, and what the schedule of feedback was—the dynamic process for behavioral intervention. The problem is rooted in the structure of research cultures in which researchers are often very siloed—a single researcher or team does not have expertise along the entire translational pipeline, from idea generation to proof of concept to broad implementation. This section of the summary highlights the overarching points raised about study design and addresses the challenges of creating and maintaining a research pipeline that has the necessary continuity between research approaches as a research program moves through the maturational process discussed by John Whyte and summarized in Chapter 3.

Organization and Pace of the Maturational Process

Individual participants offered several bird’s-eye perspectives on research design. These options were tailored to the challenge of addressing

Suggested Citation:"4 Key Themes." Institute of Medicine. 2013. Cognitive Rehabilitation Therapy for Traumatic Brain Injury: Model Study Protocols and Frameworks to Advance the State of the Science: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18257.
×

complex health problems with a complex set of interventions, as is the case for using CRT for TBI.

Dismantling vs. Additive

A “dismantling” approach begins with a large, complex problem and works with the individual components of intervention to determine which ones are essential. An “additive” approach, in contrast, begins with the individual components known to be effective and then combines them in a package. The additive approach requires that small improvements from focused interventions be accepted as progress; they may later be combined into a program that includes an algorithm to determine which components are provided to a given patient.

Using Dismantling and Additive Approaches Simultaneously

Several participants were in favor of doing top-down and bottom-up research simultaneously, a view held usually in an effort to balance urgency with building an evidence base.

In addition, one participant brought up the idea of adopting the venture capital model of disruptive innovation, suggesting that the research community may want to consider a nonlinear approach to the complex problem of using CRT for TBI. A model of disruptive innovation could be put to work to search for the one-in-a-million intervention that changes the paradigm and profoundly alters the standard of care; it could be combined with a model of incremental improvement.1

When to Optimize?

Looking more closely at the movement of a research program along the maturational process, participants examined the question of when a nascent study is ready to be optimized. They explored the question faced by researchers—at the conclusion of a study—of whether to move forward in the maturational process (toward larger studies) or to continue examining the intervention to try to increase the effect size and determine what the active ingredients are. When researchers choose the former, it means investigating whether an intervention that works in one place also works more broadly (more patients, more facilities), and this requires freezing

_____________________

1“Disruptive innovation” is a concept used in the business world, referring to new companies entering new markets with a lower-priced product related to higher-priced products in more lucrative markets. The application of this concept to health care research may revolve around seeing treatments as either new markets or new products—the possibility of trying many low-cost, unconventional interventions in an attempt to stumble somewhat blindly onto the desired combination of inexpensive and effective.

Suggested Citation:"4 Key Themes." Institute of Medicine. 2013. Cognitive Rehabilitation Therapy for Traumatic Brain Injury: Model Study Protocols and Frameworks to Advance the State of the Science: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18257.
×

the intervention and studying it in broader populations and delivered by more practitioners. When researchers opt for the latter, it means spending more time ensuring that the intervention warrants broader testing, increasing its impact on patient health, and determining which of the ingredients are delivering the effect and whether different ingredients deliver different elements of it.

The considerations underlying this choice are both scientific and pragmatic. One scientific consideration is the effect size. If the effect size is very large (or just significant and the intervention is inexpensive), researchers may opt to take it to the next point in the maturational process. If the effect size is only borderline and the intervention is costly, this might indicate the wisdom of improving the intervention’s impact before proceeding. In many cases, both forces are acting on the decision (the need to have an implementable intervention and the need for more data on a promising one), and the question becomes one of priority and resources.

Pragmatic concerns include the cost of the research and of the intervention itself and how cost compares to the magnitude of the effect and the level of need among the patient population. If an intervention is very expensive, the researcher may be satisfied with the effect size but may want to increase the impact so the intervention can be delivered, for example, less frequently or more quickly. If the intervention is relatively inexpensive and relatively effective, the researcher may choose to optimize it immediately.

SEVERITY LEVELS OF TBI: DISCUSSION OF MILD INJURY

Workshop participants individually offered suggestions and ideas concerning the needs specific to mild TBI, the form of TBI affecting the largest number of military personnel.

Definitions and Distinctions

Mild, Moderate, and Severe

A participant raised the question of how the current classification of mild, moderate, and severe may affect research outcomes. Others raised the question using a typology of symptoms to characterize people rather than the current classification system. One participant expressed the opinion that the definition of mild brain injury is the weakest part of the research agenda and that it limits the research community’s ability to make progress in the treatment of TBI.

Suggested Citation:"4 Key Themes." Institute of Medicine. 2013. Cognitive Rehabilitation Therapy for Traumatic Brain Injury: Model Study Protocols and Frameworks to Advance the State of the Science: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18257.
×

Distinction Between Mild TBI from Which an Individual Recovers and Mild TBI That Leads to Chronic Symptoms

Another participant spoke about the distinction between mild injuries from which a person recovers quickly and mild injuries that are followed by chronic complaints and a longer presentation of symptoms. She encouraged the group not to think of these two as existing along a definite continuum but rather as discrete situations. She discussed how when military service members suffer mild TBIs in combat, often the more quickly they receive interventions to mitigate the damage of the injury and see an expectation of recovery, the more likely they are to be able to return to duty within a short time. Therefore, it is desirable to have early models of intervention to prevent long-term effects, including a “disabled mentality.” She noted that a second factor affecting whether an injury will become chronic is an individual’s comorbidities. Thus, given that not all mild brain injuries have serious, long-term consequences, and given that it is possible that some multipronged interventions could make an individual’s condition worse, she believed that it is important to develop a prognostic framework for mild TBI including the role played by comorbidities.

The Origins of the Functional Deficit, and Keeping It Distinct from a Cognitive Impairment

One participant cautioned against assuming that the cause of a cognitive difficulty is a cognitive impairment. She highlighted that a practitioner treating a person with a brain injury is not necessarily treating the brain injury per se, but is treating the cognitive deficit. The deficit may be caused by factors other than an actual cognitive impairment—for example, pain. Also emphasized was the distinction between symptoms (regardless of etiology) and a documented deficit, and the importance for a given study of an intervention of determining whether or not symptoms and deficits need to be teased apart.

Closing the Knowledge Gap

The participants discussed the development of an action plan for mild TBI that would close the many gaps identified in the Institute of Medicine (IOM) report Cognitive Rehabilitation Therapy for Traumatic Brain Injury: Evaluating the Evidence. Such an action plan might fall on the “immediate” end of the spectrum since the clinical need is so great. One participant suggested that the research community may have been unduly distracted by the severe clinical needs of patients with moderate-to-severe TBI, to the detriment of research on mild injuries, which are much more common. He

Suggested Citation:"4 Key Themes." Institute of Medicine. 2013. Cognitive Rehabilitation Therapy for Traumatic Brain Injury: Model Study Protocols and Frameworks to Advance the State of the Science: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18257.
×

suggested that the research community consider beginning with the patients with milder injuries, learn how to develop effective treatments, and extrapolate to treating patients with more severe deficits.2

Several participants discussed how studies of interventions that have low-grade (but very important) effects will need to be very large and the intervention will need to have a strong impact, if the signal is to be separated from the “noise” of natural recovery. One participant recommended a grassroots approach to arriving at initial standardization (manualization). Since mild TBI is associated with a combination of issues (e.g., sensory, cognitive, and psychological problems, and symptoms such as fatigue and pain), it may be undesirable to separate out cognitive impairment too early. It can be difficult to compile a complete picture of the condition of someone with mild TBI unless comorbidities and environmental factors are taken into account. One option would be to solicit clinical impressions of effectiveness (from patient accounts and standardized measures), particularly around areas of transition in recovery, and then attempt to standardize and manualize the interventions—still in multi-ingredient form—and test them on a broader scale. (This constitutes an examples of the “dismantling” approach discussed earlier.)

COMPONENTS OF RESEARCH STUDIES AND THEIR CHARACTERISTICS AT DIFFERENT POINTS IN THE MATURATIONAL PROCESS

Below is a thematic summary of suggestions offered by individual workshop participants in either the reporting-back sessions following small-group discussions or in discussions after the formal presentations. All suggestions originated with one or several participants, and no views expressed here represent a consensus.

Treatment Theory and Enablement Theory

It is important that a specific treatment theory be articulated in a research report, raising questions of active and essential ingredients and the mechanism of action, all of which help the study to provide a stronger basis for future studies that build on it.

_____________________

2Interventions for mild TBI may not always generalize to moderate and severe TBI, and this should be considered in study design and interpretation.

Suggested Citation:"4 Key Themes." Institute of Medicine. 2013. Cognitive Rehabilitation Therapy for Traumatic Brain Injury: Model Study Protocols and Frameworks to Advance the State of the Science: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18257.
×

Study Design

Different types of questions lend themselves to different types of analysis. One participant noted that when the target of treatment is at the activity and participation level, and when the intervention is very broadly based, a health services implementation approach might be optimal. In contrast, a very focused intervention, such as attention process training, would be a good candidate for a more traditional, classical research design. Individual participants posited the following suggestions:

  • Pre-post or single-subject designs are useful.
  • Mixed methods—qualitative/quantitative—can be useful.
  • Observational studies could be used more commonly as an initial phase of research.
  • To test specific active ingredients, a design that is tighter than observational would be needed.
  • Factorial designs could be considered to compare treatment packages in order to gain power with a smaller sample size (as long as there are no adverse effects between the interventions).

Regarding studies of compensatory strategies specifically, individual participants noted the need to separately evaluate the acquisition of the strategy and the impact of using the strategy on the target domain. One participant noted how in some cases the strategies can be commercializable products. Thinking of them as products can be beneficial because it obligates the researcher to test not only efficacy but also usability and acceptability, and to think about how the product is presented—for example, on a card, a notebook, or a smartphone application. Having commercializable products also opens up new funding streams, such as Small Business Innovation Research and Small Business Technology Transfer grants.

Overall, individual participants suggested the following study design elements as the most useful or advantageous:

  • Outcomes that are meaningful to the patient
  • Interventions that are or could be manualizable
  • Designs that are flexible and expandable across populations (e.g., severity of injury)
  • Relatively short studies that are not resource intensive and are easily built upon
  • Designs with easy application to military settings (e.g., goal-management training)
  • Interventions that are easily adapted to other settings
Suggested Citation:"4 Key Themes." Institute of Medicine. 2013. Cognitive Rehabilitation Therapy for Traumatic Brain Injury: Model Study Protocols and Frameworks to Advance the State of the Science: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18257.
×

For early-stage research, some participants suggested that case reports and small studies can be valuable. They are inexpensive and can potentially detect a “signal” that subsequent studies can follow up on. Even though the focus of an early study is on impairment-level function and proximal outcomes, it is important to consider generalizability as well—tracking whether improving impairment-level function has any effect on even slightly more general functions.

For mid-stage research, participants suggested that adaptive types of designs be considered to obtain information in real time and alter an arm of a randomized clinical trial. They proposed that factorial designs be considered to compare treatment packages and thus gain power with smaller sample sizes (as long as there are not any adverse effects between the interventions). To simultaneously change the study design and broaden the population, a research team could either carry out a randomized clinical trial with three arms or do two sequential studies. The trial would have a control (standard of care) and two variables, each of which could be compared to the standard of care and to one another.

For late-stage and multisite research, individual workshop participants proposed that adaptive types of designs be considered to obtain information in real time and alter an arm of a randomized clinical trial and that factorial designs be considered to compare treatment packages and thus gain power with smaller sample sizes (as long as there are not any adverse effects between the interventions). They discussed that delayed-start designs, in which two randomized groups start in a staggered fashion, can also be beneficial. At this late stage of effectiveness research, it may be increasingly difficult to do randomized clinical trials. It may be necessary to rely more heavily on health services research, and as the Military Health System considers introducing procedures or programs, one thing they could consider is introducing them systematically and in staggered fashion such that we get naturalistic evidence of the impact of those introductions. For large-scale studies to yield key information, an essential component is a set of outcome measures that are robust, sensitive, and easily implemented in the field. Multisite networks have an advantage for heterogeneous populations in which patients often have difficulty participating (e.g., transportation to site). Multisite networks make it possible to optimize the power of the network; they also encourage standardization and generalizability of results. If a study is being done at multiple sites, it needs a measure of the quality of treatment provision, that is, the delivery of ingredients (which requires us to define the ingredients that must be delivered). A protocolized manual is required.

Suggested Citation:"4 Key Themes." Institute of Medicine. 2013. Cognitive Rehabilitation Therapy for Traumatic Brain Injury: Model Study Protocols and Frameworks to Advance the State of the Science: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18257.
×

Patient Inclusion/Exclusion

Discussion among the participants denoted that it is important to characterize the specific deficits of patients, their social support systems, and covariates (e.g., education, intact memory). Participants identified the several advantages of heterogeneity or homogeneity of the patient population at different points in the maturational process. For early- to mid-stage research, it can be beneficial for a follow-up study to use a more heterogeneous sample, one that includes comorbidities different than those in the earlier study. After a small heterogeneous study, it can also be beneficial to move to a more homogeneous group for variables such as diagnosis, severity level, type of injury, and age. To ensure that a study uses a representative patient population that could still plausibly benefit from the intervention, it may be necessary to broaden the heterogeneity of the patient population. Recruitment efforts should be diversified in order to include patients other than those referred by physicians. Finally, it is important to match the patient characteristics to the outcome(s) being evaluated (i.e., who the intervention is good for, and at what point in time will they benefit from it?).

Interventions

Individual participants remarked that it is important that an intervention be delivered uniformly and that the study design ensures that uniform delivery take place. For interventions with multiple components, it is important to determine which component contains the essential ingredient. For interventions to have the desired effect, there must not be preconditions that prevent the effect (other impairments than the one under study). If there are preconditions, then additional interventions might be necessary to address them as well. If a study is being conducted at multiple sites, it needs a measure of the quality of treatment provision—delivery of ingredients (which requires us to define the ingredients that must be delivered in a protocolized manual). Flexible interventions are advantageous in early studies as a basic approach; if researchers see a benefit in one patient population and go on to study different populations, they will be able to implement a flexible intervention relatively easily. Finally, “treatments” discussed in a paper are often technically more than one treatment. For example, when a treatment is compensatory, there are two components: the selection of the device and the use of the device. The success of either of these components is dependent on several variables having to do with the practitioner, the patient, and the treatment.

Suggested Citation:"4 Key Themes." Institute of Medicine. 2013. Cognitive Rehabilitation Therapy for Traumatic Brain Injury: Model Study Protocols and Frameworks to Advance the State of the Science: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18257.
×

Targets

Impairment-based research is more straightforward (researchers can make inferences more directly) than research on a broader disability. Workshop participants pointed out that it is important to target an impairment as opposed to a task or an entire domain. Interventions that target a task are less than ideal because they do not lend themselves to generalization. Interventions that target a domain (e.g., executive function) are less than ideal because they are so broad as to make it difficult to describe outcomes. Therefore, it is preferable that interventions target a specific functional impairment. Such functionally based interventions are generally more useful than interventions that “train to the outcome.”

Studies in which the target is a very focused impairment-level problem may lend themselves to a randomized clinical trial much more easily than studies in which the target is broadly based. For studies in which the target is very broad, researchers could consider an implementation-style research program in which the intervention is rolled out and its effects looked at broadly. Simultaneously, the effect of the components of the broad study could be examined more specifically.

Outcomes

Select participants highlighted the importance of measuring and having data on patient-centered outcomes. The IOM report Cognitive Rehabilitation Therapy for Traumatic Brain Injury: Evaluating the Evidence found a relative paucity of research on patient-centered outcomes. Although such research is not expected before efficacy has been established, it is important thereafter. As research moves through the maturational process, research on patient-centered outcomes becomes increasingly important. Consequently, it is important to do the following:

  • Test the durability of effects, use proximal outcomes.
  • Identify premorbid factors that might influence people’s initial ability to perform a task (such as set a goal).
  • Assess the ecological validity of the outcomes—the real-world impact.
  • Include performance-based measures.

Regarding outcome measures using patient reporting vs. objective measures, some studies use one and would benefit from the other. Participants noted that it is important to rate outcomes to the people who matter, (e.g., patients, caregivers, family members). It is important to not alter the outcomes

Suggested Citation:"4 Key Themes." Institute of Medicine. 2013. Cognitive Rehabilitation Therapy for Traumatic Brain Injury: Model Study Protocols and Frameworks to Advance the State of the Science: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18257.
×

being assessed when a research program moves from testing efficacy to testing effectiveness.

The participants also discussed the issue of how many outcomes are appropriate to test at the same time. Some participants thought that testing multiple outcomes is best, as they can reflect different parts of the spectrum from proximal to distal. Other participants wondered whether it were possible for a single study to have too many outcomes, and whether it might be better to limit outcomes to just one. One group discussed whether it might be best to have multiple outcomes in the early stages of research, which are honed down as research progresses, or whether it is optimal to be specific and selective at the beginning, followed by a broadening of outcomes. The further a study is from having a one-to-one correspondence between intervention and target, the greater the importance of looking either for patients who have only one problem or for packages of treatments that address the multiple problems of a patient, of which attention (or one of the other domains) is just one of the patient’s problems.

Controls

Participants discussed the importance of establishing control groups based on the treatment theory and its mechanism of action. Because rehabilitation involves helping people who are suffering and researchers do not want to give a patient nothing, they often do the following (which yields little useful information): A intervention and B intervention are compared, B shows no difference in effect from A, and it is thus not possible to know whether both are effective or neither is. Participants acknowledged that identifying an appropriate control can be difficult, for example, controlling for social interaction when the treatment is for social interaction. Consequently, it may sometimes be best to control for standard of care, unless very few people receive it. However, having standard of care as the control does make recruitment very difficult; therefore, wait list controls may offer a potentially better alternative.

Manuals

Workshop participants identified the great need for the protocols and manuals necessary as a starting point to further investigate specific interventions for specific patient populations and to examine interventions’ cost-effectiveness. The American Congress of Rehabilitation Medicine has published a cognitive rehabilitation training manual, which at this time is not specific enough to guide a practitioner’s treatment of a specific individual. For early, single-center studies, a fully developed manual is likely not warranted. Rather, a more “lightweight” manual could be used that guides

Suggested Citation:"4 Key Themes." Institute of Medicine. 2013. Cognitive Rehabilitation Therapy for Traumatic Brain Injury: Model Study Protocols and Frameworks to Advance the State of the Science: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18257.
×

the research team. The team may improve on it somewhat, but expending concentrated energy on the manual would wait until multicenter trials are being planned. To start an investigation, the standard of care could be manualized and then tested. One participant cautioned, however, about the possible negative effect on compliance of giving practitioners a manual, sending them out to do the treatments, and then revising from there.

DATABASES

Individual participants noted there are databases or repositories that exist or that are in the process of formation that may be useful as (1) potential sources of data, (2) may be repositories to which research could contribute, and (3) may assist researchers as they design studies and interpret findings. The National Institute on Disability and Rehabilitation Research TBI Model System’s National Database is available to the public. Deidentified data are provided to outside researchers who have an interest, and researchers often have the opportunity to collaborate with those who have contributed data to the TBI Model System Centers themselves. The National Institute on Disability and Rehabilitation Research has interagency agreements with the Centers for Disease Control and Prevention and other federal agencies for the use of the database to answer key research questions, which will allow for national estimates on outcomes for diverse interventions.

In parallel with the aforementioned database, the Department of Veterans Affairs has established a database that tracks people with TBI who have been admitted for inpatient rehabilitation to four of its polytrauma rehabilitation centers. This database collects all of the same data as in the TBI Model System’s National Database, as well as additional data. It will be possible, at a future date, to do comparative research across the two databases. The Common Data Element project is a collaboration among the Department of Veterans Affairs, Department of Defense, National Institutes of Health, Centers for Disease Control and Prevention, and National Institute on Disability and Rehabilitation Research. The project collects data on patient characteristics, treatments, and outcomes. Its main strength at this point is for acute care. A participant recommended that efforts coming out of this workshop build on this project. An international conversation has been going on for two to three years about the possibility of collecting high-quality, granular data across centers that would embrace the complexity of CRT for TBI—specifically for the acute stage. Such an effort would support comparative effectiveness research, allowing researchers to tease apart questions about who certain interventions work for, what treatments seem to be most effective, and the role of comorbidities.

Suggested Citation:"4 Key Themes." Institute of Medicine. 2013. Cognitive Rehabilitation Therapy for Traumatic Brain Injury: Model Study Protocols and Frameworks to Advance the State of the Science: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18257.
×

WHO OWNS THIS PROCESS GOING FORWARD?

At several points during the workshop, individual participants expressed their appreciation for the points raised, and they asked about the next steps toward having the ideas acted upon. Who owns the new understanding of the gaps in knowledge, who will formulate an action plan, and who will set forth the expectation that the action plan be carried out? Various participants expressed the view that the process is owned by multiple stakeholders, including military and other federal government agencies, and civilian researchers. Participants noted that the question of “who” looks different when referring to the military system versus the civilian world of health care. Regarding the Military Health System, John Davison3 noted that the military health and research units are many and dispersed, and it was his hope that the framework and suggestions laid out in the workshop would propel a continued dialogue—not only within the Department of Defense, but also among the Department of Defense, the Department of Veterans Affairs, and subject-matter experts—about the research necessary to move forward the use of CRT for TBI.

A participant noted what he considered to be an advantage had by the Military Health System, namely, that the standardization of care may be more feasible there than in the civilian sector. He expressed the opinion that the military system could be better able to establish standardized treatments, and this standardization could serve as a baseline that researchers improve upon. Another participant echoed these sentiments, mentioning the numerous sites operated by the Defense and Veterans Brain Injury Center (DVBIC), the Department of Veterans Affairs, and the planned satellites of the National Intrepid Center of Excellence as fertile ground for the manualized application of CRT interventions for TBI. In later stages of the maturational process of research on CRT for TBI, it may be necessary to rely more heavily on health services research. As the Military Health System considers introducing CRT interventions, one strategy it may want to consider is introducing CRT interventions systematically and in staggered fashion, thus obtaining naturalistic evidence of the impact of those introductions. A tempering viewpoint, however, was offered by Davison in his closing remarks, as he noted that military health care organizations experience much the same fragmentation as do civilian ones.

The role of funding agencies was raised multiple times over the course of the workshop. Some participants spoke of elements of a misalignment between what funders tend to fund and what researchers need funding for in order to methodically move research along the translational pipeline.

_____________________

3Acting Director, TRICARE Management Activity/Office of the Chief Medical Officer, Behavioral Medicine Division.

Suggested Citation:"4 Key Themes." Institute of Medicine. 2013. Cognitive Rehabilitation Therapy for Traumatic Brain Injury: Model Study Protocols and Frameworks to Advance the State of the Science: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18257.
×

Studies essential for the research maturational process include proof-of-concept studies, then efficacy studies, and ultimately effectiveness research; however, some in the group expressed concern that proof-of-concept studies were difficult to fund. One participant mentioned a discrepancy that sometimes exists between researchers’ and grant review panels’ views on control groups; namely, even if a research design, in order to be scientifically rigorous, calls for a control group that receives no treatment, researchers hesitate to present such a design in grant applications. At the conclusion of the workshop, John Whyte led a discussion in which he outlined three broad research strategies that could guide research decisions and help decision makers balance urgency, evidence, and cost-effectiveness.

Immediate Implementation

One strategy that the military could consider would be to take a manualized treatment shown to have significant effects in patients’ daily lives, implement it, and fund studies to elucidate the active and essential ingredients and mechanism of action. If the decision were made to implement CRT interventions immediately even while the evidence supporting their effectiveness was less than solid (i.e., only moderate amounts of evidence exist), what would this mean?

Careful Selection of the Intervention

The intervention would be selected based on a carefully considered combination of existing evidence and clinical importance. Factors to be weighed would include the magnitude and importance of the problem—how many members of the military suffer from it? One might opt against implementing an intervention with somewhat stronger evidence but that addresses a problem that few patients have, in favor of implementing an intervention that has weaker evidence but that addresses a very common problem.

Standardization of the Intervention

The intervention would need to be standardized and operationalized even if it were still unknown whether the treatment was effective; it would be crucial that researchers precisely define that which they are studying. The committee charged with the original IOM report found it difficult to determine what a given intervention consisted of; therefore, more complete, precise definitions going forward will greatly support forward movement toward fuller knowledge of different interventions. Manualization would allow researchers to assess consistency of delivery (e.g., same name of

Suggested Citation:"4 Key Themes." Institute of Medicine. 2013. Cognitive Rehabilitation Therapy for Traumatic Brain Injury: Model Study Protocols and Frameworks to Advance the State of the Science: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18257.
×

intervention, same services, same billing code), as well as systematically manipulate different elements of the intervention over time.

One advantage of this approach, in practical terms, would be that if a researcher is able to demonstrate a practical benefit from an intervention, he or she may be better able to secure funding for subsequent studies focused on the nuances of the intervention—the mechanism of action and active and essential ingredients. Another potential advantage is that it may not only advance knowledge about a specific intervention but could also foster a climate of forward movement for CRT for TBI overall.

A Middle Way: A Quasi-Experimental Health Services Design Component

A second strategy described by Whyte was a middle road, a path that beckoned somewhat less to the immediate need to implement interventions and leaned more toward establishing more robust evidence for an intervention’s effectiveness.

Selection of the Intervention

In this case the intervention would be selected based also on a combination of existing evidence and the research target’s clinical importance, in this instance putting somewhat more weight on the desirability of empirical evidence supporting an intervention’s effectiveness. The intervention would be standardized and operationalized as described above.

Experimental Design

One consideration for research design would be to assess what outcome measures are optimal to reveal the effects of the intervention. The level of function of the outcome would need to be measured carefully. For example, to measure a high-level outcome, such as returning to work, it would not be appropriate for an intervention to address a specific impairment (such as a memory problem) that is only one of many impairments associated with the high-level outcome.

The intervention might be implemented in a staggered fashion, which would allow researchers to see whether the phasing of impact matches the phasing of implementation.

It would be important to ensure that what is being measured were the results of the treatment’s implementation, and not patient variation.

Suggested Citation:"4 Key Themes." Institute of Medicine. 2013. Cognitive Rehabilitation Therapy for Traumatic Brain Injury: Model Study Protocols and Frameworks to Advance the State of the Science: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18257.
×

A Research-Heavy, Conservative Approach

A third strategy would lean most strongly in favor of rigorously establishing the effectiveness of an intervention and moving conceptually sound and promising treatments along the translational pipeline. Such an approach might involve the following:

  • Carry out smaller maturational studies aimed specifically at gaining more evidence for effectiveness—identify the active and essential ingredients.
  • Pay explicit attention to treatment ingredients, attempts to operationalize them, and attempts to measure their delivery.
  • Implement interventions as the evidence mounts.
  • Continue with quasi-experimental health services research to investigate “routine delivery,” because the evidence would have been obtained in controlled conditions and the intervention would then need to be used by a variety of practitioners in a variety of institutions and settings.

CLOSING REMARKS

John Whyte urged all participants—researchers and health services administrators in the civilian and military worlds—to continue considering what directions they think will be most productive, where resources should be directed, and what the opportunities are, given whom they treat and for what health concerns. What elements of common research strategies have hindered clear progress toward solid evidence in favor of specific CRTs? What research strategies promise to increase momentum? As the Military Health System considers implementing interventions that hold the promise of improving patients’ lives, where should it focus, and why?

The question for health services administrators and practitioners of CRT, in Whyte’s view, comes down to finding an optimal way to make decisions—the published evidence is not strong enough to guide decisions singlehandedly. Decision makers will need to sort through the pros and cons of the early implementation of treatments versus continued, more formal research, given variables such as the range of evidence and the prevalence and severity of the problem, the cost of interventions, and the cost of the research.

John Davison offered his thanks to the workshop participants, saying that the workshop had been very instructive for himself and his colleagues from TRICARE, and he expressed the hope that the dialogue would continue. He drew parallels between the process of arriving at manualized treatment for post-traumatic stress disorder (PTSD), which has been accomplished,

Suggested Citation:"4 Key Themes." Institute of Medicine. 2013. Cognitive Rehabilitation Therapy for Traumatic Brain Injury: Model Study Protocols and Frameworks to Advance the State of the Science: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18257.
×

and the process of arriving at manualized treatment for TBI, which is still very much under way. In the case of PTSD, he noted how external pressure had been important for moving the process forward; the Military Health System benefitted from outside voices asking why they were not using the most evidence-based approaches.

Addressing a question that came up several times during the workshop—that of who owns this process of advancing the state of the field—Davison acknowledged that just as researchers have a diversity of interests and concerns, so do the various branches of the organizations concerned with TBI among military service members. Centers of Excellence, research offices, clinical proponency offices, and others are “all touching different parts of the elephant.” He related the need expressed by leaders of the military health research program, namely, clinical practice guidelines for using CRT to treat TBI. In concrete terms, he noted how the original IOM report and the current workshop will be of great help to TRICARE and the Department of Defense as it makes more informed, focused program announcements in the future on CRT research directions. Finally, Davison reiterated the commitment of TRICARE to helping service members who suffer from TBIs of any type, and he asserted that the responsibility is on TRICARE’s systems to better coordinate efforts and, ultimately, arrive at clinical guidelines for using CRT to treat TBI in a way that is effective, evidence-based, and the most efficient use of dollars possible.

Suggested Citation:"4 Key Themes." Institute of Medicine. 2013. Cognitive Rehabilitation Therapy for Traumatic Brain Injury: Model Study Protocols and Frameworks to Advance the State of the Science: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18257.
×

This page is blank

Suggested Citation:"4 Key Themes." Institute of Medicine. 2013. Cognitive Rehabilitation Therapy for Traumatic Brain Injury: Model Study Protocols and Frameworks to Advance the State of the Science: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18257.
×
Page 27
Suggested Citation:"4 Key Themes." Institute of Medicine. 2013. Cognitive Rehabilitation Therapy for Traumatic Brain Injury: Model Study Protocols and Frameworks to Advance the State of the Science: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18257.
×
Page 28
Suggested Citation:"4 Key Themes." Institute of Medicine. 2013. Cognitive Rehabilitation Therapy for Traumatic Brain Injury: Model Study Protocols and Frameworks to Advance the State of the Science: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18257.
×
Page 29
Suggested Citation:"4 Key Themes." Institute of Medicine. 2013. Cognitive Rehabilitation Therapy for Traumatic Brain Injury: Model Study Protocols and Frameworks to Advance the State of the Science: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18257.
×
Page 30
Suggested Citation:"4 Key Themes." Institute of Medicine. 2013. Cognitive Rehabilitation Therapy for Traumatic Brain Injury: Model Study Protocols and Frameworks to Advance the State of the Science: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18257.
×
Page 31
Suggested Citation:"4 Key Themes." Institute of Medicine. 2013. Cognitive Rehabilitation Therapy for Traumatic Brain Injury: Model Study Protocols and Frameworks to Advance the State of the Science: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18257.
×
Page 32
Suggested Citation:"4 Key Themes." Institute of Medicine. 2013. Cognitive Rehabilitation Therapy for Traumatic Brain Injury: Model Study Protocols and Frameworks to Advance the State of the Science: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18257.
×
Page 33
Suggested Citation:"4 Key Themes." Institute of Medicine. 2013. Cognitive Rehabilitation Therapy for Traumatic Brain Injury: Model Study Protocols and Frameworks to Advance the State of the Science: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18257.
×
Page 34
Suggested Citation:"4 Key Themes." Institute of Medicine. 2013. Cognitive Rehabilitation Therapy for Traumatic Brain Injury: Model Study Protocols and Frameworks to Advance the State of the Science: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18257.
×
Page 35
Suggested Citation:"4 Key Themes." Institute of Medicine. 2013. Cognitive Rehabilitation Therapy for Traumatic Brain Injury: Model Study Protocols and Frameworks to Advance the State of the Science: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18257.
×
Page 36
Suggested Citation:"4 Key Themes." Institute of Medicine. 2013. Cognitive Rehabilitation Therapy for Traumatic Brain Injury: Model Study Protocols and Frameworks to Advance the State of the Science: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18257.
×
Page 37
Suggested Citation:"4 Key Themes." Institute of Medicine. 2013. Cognitive Rehabilitation Therapy for Traumatic Brain Injury: Model Study Protocols and Frameworks to Advance the State of the Science: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18257.
×
Page 38
Suggested Citation:"4 Key Themes." Institute of Medicine. 2013. Cognitive Rehabilitation Therapy for Traumatic Brain Injury: Model Study Protocols and Frameworks to Advance the State of the Science: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18257.
×
Page 39
Suggested Citation:"4 Key Themes." Institute of Medicine. 2013. Cognitive Rehabilitation Therapy for Traumatic Brain Injury: Model Study Protocols and Frameworks to Advance the State of the Science: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18257.
×
Page 40
Suggested Citation:"4 Key Themes." Institute of Medicine. 2013. Cognitive Rehabilitation Therapy for Traumatic Brain Injury: Model Study Protocols and Frameworks to Advance the State of the Science: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18257.
×
Page 41
Suggested Citation:"4 Key Themes." Institute of Medicine. 2013. Cognitive Rehabilitation Therapy for Traumatic Brain Injury: Model Study Protocols and Frameworks to Advance the State of the Science: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18257.
×
Page 42
Suggested Citation:"4 Key Themes." Institute of Medicine. 2013. Cognitive Rehabilitation Therapy for Traumatic Brain Injury: Model Study Protocols and Frameworks to Advance the State of the Science: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18257.
×
Page 43
Suggested Citation:"4 Key Themes." Institute of Medicine. 2013. Cognitive Rehabilitation Therapy for Traumatic Brain Injury: Model Study Protocols and Frameworks to Advance the State of the Science: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18257.
×
Page 44
Next: References »
Cognitive Rehabilitation Therapy for Traumatic Brain Injury: Model Study Protocols and Frameworks to Advance the State of the Science: Workshop Summary Get This Book
×
Buy Paperback | $35.00 Buy Ebook | $28.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

In October 2011, the Institute of Medicine (IOM) released the report Cognitive Rehabilitation Therapy for Traumatic Brain Injury: Evaluating the Evidence, assessing the published evidence for the effectiveness of using cognitive rehabilitation therapy (CRT) to treat people with traumatic brain injury (TBI). TBI has gained increasing attention in the past 15 years because of its status as the signature wound of American military conflicts in Iraq and Afghanistan. Growing numbers of U.S. service members are suffering traumatic brain injuries and are surviving them, given that (a) the majority of traumatic brain injuries are mild and (b) lifesaving measures for more severe injuries have significantly improved. People with any level of injury can require ongoing health care in their recovery, helping them to regain (or compensate for) their losses of function and supporting their full integration into their social structure and an improved quality of life.

One form of treatment for TBI is CRT, a systematic, goal-oriented approach to helping patients overcome cognitive impairments. The Department of Defense (DoD) asked the IOM to evaluate CRT for traumatic brain injury in order to guide the DoD's use and coverage in the Military Health System. Cognitive Rehabilitation Therapy for Traumatic Brain Injury: Evaluating the Evidence was the IOM's resulting study of the evidence. The report's conclusions revolved around the fact that there is little continuity among research studies of the effectiveness of different types of CRT, and there exist only small amounts of evidence (or, in many cases, none) demonstrating the effectiveness of using CRT to treat TBI—although the evidence that does exist generally indicates that CRT interventions have some effectiveness.

The workshop brought together experts in health services administration, research, and clinical practice from the civilian and military arenas in order to discuss the barriers for evaluating the effectiveness of CRT care and for identifying suggested taxonomy, terminology, timing, and ways forward for CRT researchers. The workshop consisted of individuals and was not intended to constitute a comprehensive group. Select decision makers in the Military Health System and Veterans Affairs (VA) and researchers were invited to participate. The workshop was designed to spur thinking about (1) the types of research necessary to move the field forward toward evidence-based clinical guidelines, (2) what the translational pipeline looks like and what its current deficiencies are, and (3) considerations that decision makers may choose to use as they decide what research they will support and decide how they will balance the urgency of the need with the level of evidence for CRT interventions. Cognitive Rehabilitation Therapy for Traumatic Brain Injury: Model Study Protocols and Frameworks to Advance the State of the Science summarizes the happenings of the workshop.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!