4

Key Themes

After the three opening presentations set the stage for discussion, the workshop participants were divided into small groups and asked to analyze three published research papers on cognitive rehabilitation therapy (CRT) for traumatic brain injury (TBI). Workshop participants had been provided with the three papers and a list of questions for each prior to the workshop. Committee members had drafted the discussion questions, and a committee member moderated each small-group discussion. The discussion of the three papers was not intended to reassess the papers’ conclusions; rather, the discussions were designed to elicit participants’ suggestions about the elements of optimal research design and the succession of studies through the maturational process. Participants evaluated variables such as sample size, patient selection, description of interventions, and selection and measurement of outcomes, with an eye toward generalizing their assessments to inform the design of future studies. Following each small-group discussion, a member of each group reported back to the entire group and a free-ranging discussion ensued. This chapter summarizes the key themes in the report-back sessions. As was noted in Chapter 1, all ideas and suggestions expressed by participants do not reflect a consensus and should not be interpreted as such.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 27
4 Key Themes After the three opening presentations set the stage for discussion, the workshop participants were divided into small groups and asked to analyze three published research papers on cognitive rehabilitation therapy (CRT) for traumatic brain injury (TBI). Workshop participants had been provided with the three papers and a list of questions for each prior to the workshop. Committee members had drafted the discussion questions, and a commit- tee member moderated each small-group discussion. The discussion of the three papers was not intended to reassess the papers’ conclusions; rather, the discussions were designed to elicit participants’ suggestions about the elements of optimal research design and the succession of studies through the maturational process. Participants evaluated variables such as sample size, patient selection, description of interventions, and selection and mea- surement of outcomes, with an eye toward generalizing their assessments to inform the design of future studies. Following each small-group discus- sion, a member of each group reported back to the entire group and a free-ranging discussion ensued. This chapter summarizes the key themes in the report-back sessions. As was noted in Chapter 1, all ideas and sugges- tions expressed by participants do not reflect a consensus and should not be interpreted as such. 27

OCR for page 27
28 COGNITIVE REHABILITATION THERAPY FOR TBI THE TRANSLATIONAL PIPELINE AND RESEARCH DESIGN Balancing Urgency vs. Evidence Health service administrators and service providers must choose which interventions to pursue by balancing competing needs: the magnitude of patients’ needs, the magnitude of the intervention’s benefit, the cost of the research, and the cost of the intervention itself. There was a good deal of discussion about the continuum from “implement something immediately” to “wait and research interventions thoroughly.” Participants explored the question, how clear does the evidence need to be for an intervention to be worth implementing? At one end of the continuum is the decision to imple- ment interventions right away (without as strong of evidence of efficacy as might be desired) and to plan subsequent research based on what the implementation showed. At the other end of the continuum is the decision to more rigorously determine which interventions are effective and why, followed then by implementation. Individual participants expressed favor for each strategy, depending on the balance of urgency, the effect size, cost- effectiveness, and other factors. Promoting Continuity in the Research Pipeline Several participants agreed on the need for greater continuity in the translational pipeline for using CRT to treat TBI. For any given domain (attention, executive function, language and social communication, visuo- spatial perception, and memory) and patient population (severity of injury, stage of recovery), there are few published papers that describe in detail what the intervention was, what kind of feedback was measured, and what the schedule of feedback was—the dynamic process for behavioral interven- tion. The problem is rooted in the structure of research cultures in which researchers are often very siloed—a single researcher or team does not have expertise along the entire translational pipeline, from idea generation to proof of concept to broad implementation. This section of the summary highlights the overarching points raised about study design and addresses the challenges of creating and maintaining a research pipeline that has the necessary continuity between research approaches as a research program moves through the maturational process discussed by John Whyte and sum- marized in Chapter 3. Organization and Pace of the Maturational Process Individual participants offered several bird’s-eye perspectives on re- search design. These options were tailored to the challenge of addressing

OCR for page 27
KEY THEMES 29 complex health problems with a complex set of interventions, as is the case for using CRT for TBI. Dismantling vs. Additive A “dismantling” approach begins with a large, complex problem and works with the individual components of intervention to determine which ones are essential. An “additive” approach, in contrast, begins with the individual components known to be effective and then combines them in a package. The additive approach requires that small improvements from focused interventions be accepted as progress; they may later be combined into a program that includes an algorithm to determine which components are provided to a given patient. Using Dismantling and Additive Approaches Simultaneously Several participants were in favor of doing top-down and bottom-up research simultaneously, a view held usually in an effort to balance urgency with building an evidence base. In addition, one participant brought up the idea of adopting the ven- ture capital model of disruptive innovation, suggesting that the research community may want to consider a nonlinear approach to the complex problem of using CRT for TBI. A model of disruptive innovation could be put to work to search for the one-in-a-million intervention that changes the paradigm and profoundly alters the standard of care; it could be combined with a model of incremental improvement.1 When to Optimize? Looking more closely at the movement of a research program along the maturational process, participants examined the question of when a nascent study is ready to be optimized. They explored the question faced by researchers—at the conclusion of a study—of whether to move forward in the maturational process (toward larger studies) or to continue examin- ing the intervention to try to increase the effect size and determine what the active ingredients are. When researchers choose the former, it means investigating whether an intervention that works in one place also works more broadly (more patients, more facilities), and this requires freezing 1  “Disruptive innovation” is a concept used in the business world, referring to new com- panies entering new markets with a lower-priced product related to higher-priced products in more lucrative markets. The application of this concept to health care research may revolve around seeing treatments as either new markets or new products—the possibility of trying many low-cost, unconventional interventions in an attempt to stumble somewhat blindly onto the desired combination of inexpensive and effective.

OCR for page 27
30 COGNITIVE REHABILITATION THERAPY FOR TBI the intervention and studying it in broader populations and delivered by more practitioners. When researchers opt for the latter, it means spending more time ensuring that the intervention warrants broader testing, increas- ing its impact on patient health, and determining which of the ingredients are delivering the effect and whether different ingredients deliver different elements of it. The considerations underlying this choice are both scientific and prag- matic. One scientific consideration is the effect size. If the effect size is very large (or just significant and the intervention is inexpensive), researchers may opt to take it to the next point in the maturational process. If the effect size is only borderline and the intervention is costly, this might indicate the wisdom of improving the intervention’s impact before proceeding. In many cases, both forces are acting on the decision (the need to have an imple- mentable intervention and the need for more data on a promising one), and the question becomes one of priority and resources. Pragmatic concerns include the cost of the research and of the inter- vention itself and how cost compares to the magnitude of the effect and the level of need among the patient population. If an intervention is very expensive, the researcher may be satisfied with the effect size but may want to increase the impact so the intervention can be delivered, for example, less frequently or more quickly. If the intervention is relatively inexpensive and relatively effective, the researcher may choose to optimize it immediately. SEVERITY LEVELS OF TBI: DISCUSSION OF MILD INJURY Workshop participants individually offered suggestions and ideas con- cerning the needs specific to mild TBI, the form of TBI affecting the largest number of military personnel. Definitions and Distinctions Mild, Moderate, and Severe A participant raised the question of how the current classification of mild, moderate, and severe may affect research outcomes. Others raised the question using a typology of symptoms to characterize people rather than the current classification system. One participant expressed the opinion that the definition of mild brain injury is the weakest part of the research agenda and that it limits the research community’s ability to make progress in the treatment of TBI.

OCR for page 27
KEY THEMES 31 Distinction Between Mild TBI from Which an Individual Recovers and Mild TBI That Leads to Chronic Symptoms Another participant spoke about the distinction between mild injuries from which a person recovers quickly and mild injuries that are followed by chronic complaints and a longer presentation of symptoms. She encouraged the group not to think of these two as existing along a definite continuum but rather as discrete situations. She discussed how when military service members suffer mild TBIs in combat, often the more quickly they receive interventions to mitigate the damage of the injury and see an expectation of recovery, the more likely they are to be able to return to duty within a short time. Therefore, it is desirable to have early models of intervention to prevent long-term effects, including a “disabled mentality.” She noted that a second factor affecting whether an injury will become chronic is an individual’s comorbidities. Thus, given that not all mild brain injuries have serious, long-term consequences, and given that it is possible that some multipronged interventions could make an individual’s condition worse, she believed that it is important to develop a prognostic framework for mild TBI including the role played by comorbidities. The Origins of the Functional Deficit, and Keeping It Distinct from a Cognitive Impairment One participant cautioned against assuming that the cause of a cogni- tive difficulty is a cognitive impairment. She highlighted that a practitioner treating a person with a brain injury is not necessarily treating the brain injury per se, but is treating the cognitive deficit. The deficit may be caused by factors other than an actual cognitive impairment—for example, pain. Also emphasized was the distinction between symptoms (regardless of eti- ology) and a documented deficit, and the importance for a given study of an intervention of determining whether or not symptoms and deficits need to be teased apart. Closing the Knowledge Gap The participants discussed the development of an action plan for mild TBI that would close the many gaps identified in the Institute of Medicine (IOM) report Cognitive Rehabilitation Therapy for Traumatic Brain Injury: Evaluating the Evidence. Such an action plan might fall on the “immedi- ate” end of the spectrum since the clinical need is so great. One participant suggested that the research community may have been unduly distracted by the severe clinical needs of patients with moderate-to-severe TBI, to the detriment of research on mild injuries, which are much more common. He

OCR for page 27
32 COGNITIVE REHABILITATION THERAPY FOR TBI suggested that the research community consider beginning with the pa- tients with milder injuries, learn how to develop effective treatments, and extrapolate to treating patients with more severe deficits.2 Several participants discussed how studies of interventions that have low-grade (but very important) effects will need to be very large and the intervention will need to have a strong impact, if the signal is to be sepa- rated from the “noise” of natural recovery. One participant recommended a grassroots approach to arriving at initial standardization (manualization). Since mild TBI is associated with a combination of issues (e.g., sensory, cognitive, and psychological problems, and symptoms such as fatigue and pain), it may be undesirable to separate out cognitive impairment too early. It can be difficult to compile a complete picture of the condition of someone with mild TBI unless comorbidities and environmental factors are taken into account. One option would be to solicit clinical impressions of effec- tiveness (from patient accounts and standardized measures), particularly around areas of transition in recovery, and then attempt to standardize and manualize the interventions—still in multi-ingredient form—and test them on a broader scale. (This constitutes an examples of the “dismantling” ap- proach discussed earlier.) COMPONENTS OF RESEARCH STUDIES AND THEIR CHARACTERISTICS AT DIFFERENT POINTS IN THE MATURATIONAL PROCESS Below is a thematic summary of suggestions offered by individual workshop participants in either the reporting-back sessions following small- group discussions or in discussions after the formal presentations. All sug- gestions originated with one or several participants, and no views expressed here represent a consensus. Treatment Theory and Enablement Theory It is important that a specific treatment theory be articulated in a re- search report, raising questions of active and essential ingredients and the mechanism of action, all of which help the study to provide a stronger basis for future studies that build on it. 2  Interventions for mild TBI may not always generalize to moderate and severe TBI, and this should be considered in study design and interpretation.

OCR for page 27
KEY THEMES 33 Study Design Different types of questions lend themselves to different types of analy- sis. One participant noted that when the target of treatment is at the activity and participation level, and when the intervention is very broadly based, a health services implementation approach might be optimal. In contrast, a very focused intervention, such as attention process training, would be a good candidate for a more traditional, classical research design. Individual participants posited the following suggestions: • Pre-post or single-subject designs are useful. • Mixed methods—qualitative/quantitative—can be useful. • Observational studies could be used more commonly as an initial phase of research. • To test specific active ingredients, a design that is tighter than ob- servational would be needed. • Factorial designs could be considered to compare treatment pack- ages in order to gain power with a smaller sample size (as long as there are no adverse effects between the interventions). Regarding studies of compensatory strategies specifically, individual participants noted the need to separately evaluate the acquisition of the strategy and the impact of using the strategy on the target domain. One participant noted how in some cases the strategies can be commercializable products. Thinking of them as products can be beneficial because it obli- gates the researcher to test not only efficacy but also usability and accept- ability, and to think about how the product is presented—for example, on a card, a notebook, or a smartphone application. Having commercializable products also opens up new funding streams, such as Small Business Inno- vation Research and Small Business Technology Transfer grants. Overall, individual participants suggested the following study design elements as the most useful or advantageous: • Outcomes that are meaningful to the patient • Interventions that are or could be manualizable • Designs that are flexible and expandable across populations (e.g., severity of injury) • Relatively short studies that are not resource intensive and are eas- ily built upon • Designs with easy application to military settings (e.g., goal-­ management training) • Interventions that are easily adapted to other settings

OCR for page 27
34 COGNITIVE REHABILITATION THERAPY FOR TBI For early-stage research, some participants suggested that case reports and small studies can be valuable. They are inexpensive and can potentially detect a “signal” that subsequent studies can follow up on. Even though the focus of an early study is on impairment-level function and proximal outcomes, it is important to consider generalizability as well—tracking whether improving impairment-level function has any effect on even slightly more general functions. For mid-stage research, participants suggested that adaptive types of designs be considered to obtain information in real time and alter an arm of a randomized clinical trial. They proposed that factorial designs be con- sidered to compare treatment packages and thus gain power with smaller sample sizes (as long as there are not any adverse effects between the in- terventions). To simultaneously change the study design and broaden the population, a research team could either carry out a randomized clinical trial with three arms or do two sequential studies. The trial would have a control (standard of care) and two variables, each of which could be com- pared to the standard of care and to one another. For late-stage and multisite research, individual workshop participants proposed that adaptive types of designs be considered to obtain information in real time and alter an arm of a randomized clinical trial and that facto- rial designs be considered to compare treatment packages and thus gain power with smaller sample sizes (as long as there are not any adverse ef- fects between the interventions). They discussed that delayed-start designs, in which two randomized groups start in a staggered fashion, can also be beneficial. At this late stage of effectiveness research, it may be increasingly difficult to do randomized clinical trials. It may be necessary to rely more heavily on health services research, and as the Military Health System con- siders introducing procedures or programs, one thing they could consider is introducing them systematically and in staggered fashion such that we get naturalistic evidence of the impact of those introductions. For large- scale studies to yield key information, an essential component is a set of outcome measures that are robust, sensitive, and easily implemented in the field. Multisite networks have an advantage for heterogeneous populations in which patients often have difficulty participating (e.g., transportation to site). Multisite networks make it possible to optimize the power of the net- work; they also encourage standardization and generalizability of results. If a study is being done at multiple sites, it needs a measure of the quality of treatment provision, that is, the delivery of ingredients (which requires us to define the ingredients that must be delivered). A protocolized manual is required.

OCR for page 27
KEY THEMES 35 Patient Inclusion/Exclusion Discussion among the participants denoted that it is important to characterize the specific deficits of patients, their social support systems, and covariates (e.g., education, intact memory). Participants identified the several advantages of heterogeneity or homogeneity of the patient popula- tion at different points in the maturational process. For early- to mid-stage research, it can be beneficial for a follow-up study to use a more heteroge- neous sample, one that includes comorbidities different than those in the earlier study. After a small heterogeneous study, it can also be beneficial to move to a more homogeneous group for variables such as diagnosis, sever- ity level, type of injury, and age. To ensure that a study uses a representative patient population that could still plausibly benefit from the intervention, it may be necessary to broaden the heterogeneity of the patient population. Recruitment efforts should be diversified in order to include patients other than those referred by physicians. Finally, it is important to match the patient characteristics to the outcome(s) being evaluated (i.e., who the in- tervention is good for, and at what point in time will they benefit from it?). Interventions Individual participants remarked that it is important that an interven- tion be delivered uniformly and that the study design ensures that uniform delivery take place. For interventions with multiple components, it is im- portant to determine which component contains the essential ingredient. For interventions to have the desired effect, there must not be preconditions that prevent the effect (other impairments than the one under study). If there are preconditions, then additional interventions might be necessary to address them as well. If a study is being conducted at multiple sites, it needs a measure of the quality of treatment provision—delivery of ingredients (which requires us to define the ingredients that must be delivered in a pro- tocolized manual). Flexible interventions are advantageous in early studies as a basic approach; if researchers see a benefit in one patient population and go on to study different populations, they will be able to implement a flexible intervention relatively easily. Finally, “treatments” discussed in a paper are often technically more than one treatment. For example, when a treatment is compensatory, there are two components: the selection of the device and the use of the device. The success of either of these components is dependent on several variables having to do with the practitioner, the patient, and the treatment.

OCR for page 27
36 COGNITIVE REHABILITATION THERAPY FOR TBI Targets Impairment-based research is more straightforward (researchers can make inferences more directly) than research on a broader disability. Work- shop participants pointed out that it is important to target an impairment as opposed to a task or an entire domain. Interventions that target a task are less than ideal because they do not lend themselves to generalization. Interventions that target a domain (e.g., executive function) are less than ideal because they are so broad as to make it difficult to describe outcomes. Therefore, it is preferable that interventions target a specific functional im- pairment. Such functionally based interventions are generally more useful than interventions that “train to the outcome.” Studies in which the target is a very focused impairment-level problem may lend themselves to a randomized clinical trial much more easily than studies in which the target is broadly based. For studies in which the target is very broad, researchers could consider an implementation-style research program in which the intervention is rolled out and its effects looked at broadly. Simultaneously, the effect of the components of the broad study could be examined more specifically. Outcomes Select participants highlighted the importance of measuring and having data on patient-centered outcomes. The IOM report Cognitive Rehabilita- tion Therapy for Traumatic Brain Injury: Evaluating the Evidence found a relative paucity of research on patient-centered outcomes. Although such research is not expected before efficacy has been established, it is important thereafter. As research moves through the maturational process, research on patient-centered outcomes becomes increasingly important. Consequently, it is important to do the following: • Test the durability of effects, use proximal outcomes. • Identify premorbid factors that might influence people’s initial abil- ity to perform a task (such as set a goal). • Assess the ecological validity of the outcomes—the real-world impact. • Include performance-based measures. Regarding outcome measures using patient reporting vs. objective mea- sures, some studies use one and would benefit from the other. Participants noted that it is important to rate outcomes to the people who matter, (e.g., patients, caregivers, family members). It is important to not alter the out-

OCR for page 27
KEY THEMES 37 comes being assessed when a research program moves from testing efficacy to testing effectiveness. The participants also discussed the issue of how many outcomes are appropriate to test at the same time. Some participants thought that testing multiple outcomes is best, as they can reflect different parts of the spectrum from proximal to distal. Other participants wondered whether it were pos- sible for a single study to have too many outcomes, and whether it might be better to limit outcomes to just one. One group discussed whether it might be best to have multiple outcomes in the early stages of research, which are honed down as research progresses, or whether it is optimal to be specific and selective at the beginning, followed by a broadening of outcomes. The further a study is from having a one-to-one correspondence between inter- vention and target, the greater the importance of looking either for patients who have only one problem or for packages of treatments that address the multiple problems of a patient, of which attention (or one of the other domains) is just one of the patient’s problems. Controls Participants discussed the importance of establishing control groups based on the treatment theory and its mechanism of action. Because reha- bilitation involves helping people who are suffering and researchers do not want to give a patient nothing, they often do the following (which yields little useful information): A intervention and B intervention are compared, B shows no difference in effect from A, and it is thus not possible to know whether both are effective or neither is. Participants acknowledged that identifying an appropriate control can be difficult, for example, controlling for social interaction when the treatment is for social interaction. Conse- quently, it may sometimes be best to control for standard of care, unless very few people receive it. However, having standard of care as the control does make recruitment very difficult; therefore, wait list controls may offer a potentially better alternative. Manuals Workshop participants identified the great need for the protocols and manuals necessary as a starting point to further investigate specific inter- ventions for specific patient populations and to examine interventions’ cost-effectiveness. The American Congress of Rehabilitation Medicine has published a cognitive rehabilitation training manual, which at this time is not specific enough to guide a practitioner’s treatment of a specific indi- vidual. For early, single-center studies, a fully developed manual is likely not warranted. Rather, a more “lightweight” manual could be used that guides

OCR for page 27
38 COGNITIVE REHABILITATION THERAPY FOR TBI the research team. The team may improve on it somewhat, but expending concentrated energy on the manual would wait until multicenter trials are being planned. To start an investigation, the standard of care could be manualized and then tested. One participant cautioned, however, about the possible negative effect on compliance of giving practitioners a manual, sending them out to do the treatments, and then revising from there. DATABASES Individual participants noted there are databases or repositories that exist or that are in the process of formation that may be useful as (1) potential sources of data, (2) may be repositories to which research could contribute, and (3) may assist researchers as they design studies and in- terpret findings. The National Institute on Disability and Rehabilitation Research TBI Model System’s National Database is available to the public. Deidentified data are provided to outside researchers who have an inter- est, and researchers often have the opportunity to collaborate with those who have contributed data to the TBI Model System Centers themselves. The National Institute on Disability and Rehabilitation Research has inter- agency agreements with the Centers for Disease Control and Prevention and other federal agencies for the use of the database to answer key research questions, which will allow for national estimates on outcomes for diverse interventions. In parallel with the aforementioned database, the Department of Vet- erans Affairs has established a database that tracks people with TBI who have been admitted for inpatient rehabilitation to four of its polytrauma rehabilitation centers. This database collects all of the same data as in the TBI Model System’s National Database, as well as additional data. It will be possible, at a future date, to do comparative research across the two databases. The Common Data Element project is a collaboration among the Department of Veterans Affairs, Department of Defense, National Institutes of Health, Centers for Disease Control and Prevention, and National In- stitute on Disability and Rehabilitation Research. The project collects data on patient characteristics, treatments, and outcomes. Its main strength at this point is for acute care. A participant recommended that efforts coming out of this workshop build on this project. An international conversation has been going on for two to three years about the possibility of collecting high-quality, granular data across centers that would embrace the complex- ity of CRT for TBI—specifically for the acute stage. Such an effort would support comparative effectiveness research, allowing researchers to tease apart questions about who certain interventions work for, what treatments seem to be most effective, and the role of comorbidities.

OCR for page 27
KEY THEMES 39 WHO OWNS THIS PROCESS GOING FORWARD? At several points during the workshop, individual participants ex- pressed their appreciation for the points raised, and they asked about the next steps toward having the ideas acted upon. Who owns the new under- standing of the gaps in knowledge, who will formulate an action plan, and who will set forth the expectation that the action plan be carried out? Vari- ous participants expressed the view that the process is owned by multiple stakeholders, including military and other federal government agencies, and civilian researchers. Participants noted that the question of “who” looks different when referring to the military system versus the civilian world of health care. Regarding the Military Health System, John Davison3 noted that the military health and research units are many and dispersed, and it was his hope that the framework and suggestions laid out in the workshop would propel a continued dialogue—not only within the Department of Defense, but also among the Department of Defense, the Department of Veterans Affairs, and subject-matter experts—about the research necessary to move forward the use of CRT for TBI. A participant noted what he considered to be an advantage had by the Military Health System, namely, that the standardization of care may be more feasible there than in the civilian sector. He expressed the opinion that the military system could be better able to establish standardized treat- ments, and this standardization could serve as a baseline that researchers improve upon. Another participant echoed these sentiments, mentioning the numerous sites operated by the Defense and Veterans Brain Injury Center (DVBIC), the Department of Veterans Affairs, and the planned satellites of the National Intrepid Center of Excellence as fertile ground for the manualized application of CRT interventions for TBI. In later stages of the maturational process of research on CRT for TBI, it may be necessary to rely more heavily on health services research. As the Military Health System considers introducing CRT interventions, one strategy it may want to consider is introducing CRT interventions systematically and in stag- gered fashion, thus obtaining naturalistic evidence of the impact of those introductions. A tempering viewpoint, however, was offered by Davison in his closing remarks, as he noted that military health care organizations experience much the same fragmentation as do civilian ones. The role of funding agencies was raised multiple times over the course of the workshop. Some participants spoke of elements of a misalignment between what funders tend to fund and what researchers need funding for in order to methodically move research along the translational pipeline. 3 Acting Director, TRICARE Management Activity/Office of the Chief Medical Officer, Behavioral Medicine Division.

OCR for page 27
40 COGNITIVE REHABILITATION THERAPY FOR TBI Studies essential for the research maturational process include proof-of- concept studies, then efficacy studies, and ultimately effectiveness research; however, some in the group expressed concern that proof-of-concept stud- ies were difficult to fund. One participant mentioned a discrepancy that sometimes exists between researchers’ and grant review panels’ views on control groups; namely, even if a research design, in order to be scientifically rigorous, calls for a control group that receives no treatment, researchers hesitate to present such a design in grant applications. At the conclusion of the workshop, John Whyte led a discussion in which he outlined three broad research strategies that could guide research decisions and help deci- sion makers balance urgency, evidence, and cost-effectiveness. Immediate Implementation One strategy that the military could consider would be to take a manu- alized treatment shown to have significant effects in patients’ daily lives, implement it, and fund studies to elucidate the active and essential ingre- dients and mechanism of action. If the decision were made to implement CRT interventions immediately even while the evidence supporting their effectiveness was less than solid (i.e., only moderate amounts of evidence exist), what would this mean? Careful Selection of the Intervention The intervention would be selected based on a carefully considered combination of existing evidence and clinical importance. Factors to be weighed would include the magnitude and importance of the problem— how many members of the military suffer from it? One might opt against implementing an intervention with somewhat stronger evidence but that addresses a problem that few patients have, in favor of implementing an intervention that has weaker evidence but that addresses a very common problem. Standardization of the Intervention The intervention would need to be standardized and operationalized even if it were still unknown whether the treatment was effective; it would be crucial that researchers precisely define that which they are studying. The committee charged with the original IOM report found it difficult to determine what a given intervention consisted of; therefore, more complete, precise definitions going forward will greatly support forward movement toward fuller knowledge of different interventions. Manualization would allow researchers to assess consistency of delivery (e.g., same name of

OCR for page 27
KEY THEMES 41 intervention, same services, same billing code), as well as systematically manipulate different elements of the intervention over time. One advantage of this approach, in practical terms, would be that if a researcher is able to demonstrate a practical benefit from an intervention, he or she may be better able to secure funding for subsequent studies focused on the nuances of the intervention—the mechanism of action and active and essential ingredients. Another potential advantage is that it may not only advance knowledge about a specific intervention but could also foster a climate of forward movement for CRT for TBI overall. A Middle Way: A Quasi-Experimental Health Services Design Component A second strategy described by Whyte was a middle road, a path that beckoned somewhat less to the immediate need to implement interventions and leaned more toward establishing more robust evidence for an interven- tion’s effectiveness. Selection of the Intervention In this case the intervention would be selected based also on a combi- nation of existing evidence and the research target’s clinical importance, in this instance putting somewhat more weight on the desirability of empirical evidence supporting an intervention’s effectiveness. The intervention would be standardized and operationalized as described above. Experimental Design One consideration for research design would be to assess what outcome measures are optimal to reveal the effects of the intervention. The level of function of the outcome would need to be measured carefully. For example, to measure a high-level outcome, such as returning to work, it would not be appropriate for an intervention to address a specific impairment (such as a memory problem) that is only one of many impairments associated with the high-level outcome. The intervention might be implemented in a staggered fashion, which would allow researchers to see whether the phasing of impact matches the phasing of implementation. It would be important to ensure that what is being measured were the results of the treatment’s implementation, and not patient variation.

OCR for page 27
42 COGNITIVE REHABILITATION THERAPY FOR TBI A Research-Heavy, Conservative Approach A third strategy would lean most strongly in favor of rigorously es- tablishing the effectiveness of an intervention and moving conceptually sound and promising treatments along the translational pipeline. Such an approach might involve the following: • Carry out smaller maturational studies aimed specifically at gaining more evidence for effectiveness—identify the active and essential ingredients. • Pay explicit attention to treatment ingredients, attempts to opera- tionalize them, and attempts to measure their delivery. • Implement interventions as the evidence mounts. • Continue with quasi-experimental health services research to in- vestigate “routine delivery,” because the evidence would have been obtained in controlled conditions and the intervention would then need to be used by a variety of practitioners in a variety of institu- tions and settings. CLOSING REMARKS John Whyte urged all participants—researchers and health services administrators in the civilian and military worlds—to continue considering what directions they think will be most productive, where resources should be directed, and what the opportunities are, given whom they treat and for what health concerns. What elements of common research strategies have hindered clear progress toward solid evidence in favor of specific CRTs? What research strategies promise to increase momentum? As the Military Health System considers implementing interventions that hold the promise of improving patients’ lives, where should it focus, and why? The question for health services administrators and practitioners of CRT, in Whyte’s view, comes down to finding an optimal way to make decisions—the published evidence is not strong enough to guide decisions singlehandedly. Decision makers will need to sort through the pros and cons of the early implementation of treatments versus continued, more formal research, given variables such as the range of evidence and the prevalence and severity of the problem, the cost of interventions, and the cost of the research. John Davison offered his thanks to the workshop participants, saying that the workshop had been very instructive for himself and his colleagues from TRICARE, and he expressed the hope that the dialogue would con- tinue. He drew parallels between the process of arriving at manualized treatment for post-traumatic stress disorder (PTSD), which has been accom­

OCR for page 27
KEY THEMES 43 plished, and the process of arriving at manualized treatment for TBI, which is still very much under way. In the case of PTSD, he noted how external pressure had been important for moving the process forward; the Military Health System benefitted from outside voices asking why they were not using the most evidence-based approaches. Addressing a question that came up several times during the workshop— that of who owns this process of advancing the state of the field—Davison acknowledged that just as researchers have a diversity of interests and con- cerns, so do the various branches of the organizations concerned with TBI among military service members. Centers of Excellence, research offices, clinical proponency offices, and others are “all touching different parts of the elephant.” He related the need expressed by leaders of the military health research program, namely, clinical practice guidelines for using CRT to treat TBI. In concrete terms, he noted how the original IOM report and the current workshop will be of great help to TRICARE and the Depart- ment of Defense as it makes more informed, focused program announce- ments in the future on CRT research directions. Finally, Davison reiterated the commitment of TRICARE to helping service members who suffer from TBIs of any type, and he asserted that the responsibility is on TRICARE’s systems to better coordinate efforts and, ultimately, arrive at clinical guide- lines for using CRT to treat TBI in a way that is effective, evidence-based, and the most efficient use of dollars possible.

OCR for page 27