Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
6 Suggestions for Analysis Plans by Working Groups During the planning phase of the workshop, the steering committee compiled a list of 36 issues raised by stakeholders in the Military Health System (MHS) medical community related to the care of patients with mild, moderate, and severe traumatic brain injury (TBI) (see Appen- dix B). From this list, they identified the areas that could potentially benefit from operational systems engineering (OSE) approaches and categorized them into five major challenges for TBI care management: 1. Development of new TBI knowledge 2. Detection and screening of TBI conditions 3. TBI care coordination and communication 4. Measurement and forecasting of demand for TBI care 5. TBI care system capacity, organization, and resource allocation The committee then converted the stakeholder issues in these five categories into two or three issues for OSE analysis (i.e., analytical chal- lenges that, if addressed effectively through OSE approaches, would answer important questions and help improve the performance of TBI care) (see Appendix C). Forty of the 50 invited participants at the workshop (Appendixes F and G) were assigned, on the basis of their expertise, to one of the five working groups listed above. The OSE analysis issues and challenges assigned to each working group are listed below: Steering committee co-chairs Norman Augustine and Denis Cortese and steering com- mittee member Seth Bonder circulated among the five working groups. 81
82 Systems Engineering to Improve Traumatic Brain Injury CARE â¢ Group A. Development of New TBI Knowledge â¢ A.1. Develop an approach to modeling the neuropathology and clinical dynamics of blast and concussive effects on brain function that lead to mild, moderate, or severe TBI. â¢ A.2. Develop an acute-to-chronic disease model of mild trau- matic brain injury (mTBI). â¢ Group B. Detection and Screening of TBI Conditions â¢ B.1. Develop a model for diagnosing mTBI based on clinical experience and cognitive testing. â¢ B.2. Develop the structure and processes of an mTBI screen- ing program for use in theater and in the continental United States (CONUS). â¢ Group C. Coordination and Communication for TBI Care â¢ C.1. Develop the structure of a TBI information system to track, monitor, and cue patients, families, and relevant providers. â¢ C.2. Develop a methodology for coordinating the delivery of TBI care services immediately following trauma. â¢ Group D. Measuring and Forecasting the Demand for TBI Care â¢ D.1. Based on historical data, develop a statistical estimate of TBI in the population of military personnel involved in Operation Iraqi Freedom/Operation Enduring Freedom (OIF/OEF). â¢ D.2. Develop a methodology of forecasting the time stream of future TBI cases in the military population. â¢ D.3. Develop elements of an assessment and a methodology for assessing the value of preventing TBI. â¢ Group E. Capacity, Organization, and Resource Allocations for a TBI Care System â¢ E.1. Describe elements, processes, and activities to represent the dynamics of a complete course of TBI care as input for a model of a TBI care system. â¢ E.2. Outline the structure of a model or methodology to assist in planning for the allocation of scarce TBI care providers in theater and in CONUS.
Suggestions for Analysis Plans by Working Groups 83 For each challenge, the working groups were asked to design a suggestion for an analysis plan, including an objective, a technical a Â pproach, an approach structure, data requirements, critical assump- tions and constraints, metrics, expected output, implementation actions, estimates of time and resource requirements, and other elements of a future OSE study or program that might be initiated to meet the target challenge. Specifically, working groups were asked to identify the types of approaches, methods, and information that could be developed by OSE practitioners to assist care providers and managers in delivering quality TBI care. Each group had a chairperson who served as the technical lead, a rapporteur who was responsible for summarizing and communicating the technical aspects of the groupâs approach, and several experts in rel- evant subject matter. The group was asked to review its assigned tasks; modify them as appropriate; and develop analysis plans for a study, method, or means of data collection. Each group was also instructed to design its analysis plans so they could potentially be used by MHS. Group members offered individual suggestions and ideas for the develÂ opÂment of the analysis plan, and no attempt was made to Âensure a con- sensus. The chairman and rapporteur of each working group presented a summary of the group discussion to the full workshop. The five working groups were not Academy-appointed committees, and this summary reflects the views of the individuals who participated in each working groupânot necessarily those of the institution or the workshop plan- ning committee. The analysis approach to each task comprised three parts: the issue itself (what each group was asked to do and the purpose of the task); the reasons the task was addressed (in terms of the stakeholder issues on which it was based); and the analysis output (the study, approach, or method that could be developed by implementing the analysis plan, essentially describing the capabilities of a specific model based on the suggested approach). Each plan for a future OSE study or program included an objec- tive stating what could potentially be achieved through this approach; a description of the technical approach to achieving the specified ob- jective; and an approach structure, including the variables considered for inclusion, the development of necessary relationships, important statistical and structural model formulations, and a description of how the technical approach would be implemented.
84 Systems Engineering to Improve Traumatic Brain Injury CARE In addition, each group was asked to identify data that might have to be collected, as well as critical assumptions about future demand and data availability and critical constraints based on available measurement tools, sample sizes, and safety regulations. Each group also outlined metrics for evaluating performance, outcome, and utility and identified specific tasks for executing the suggested approach. Finally, each analysis plan included expected output, implementation actions, estimates of duration, and resource requirements. The following summaries of the suggestions for analysis plans are based on presentations by the chair and rapporteur of each group. It is important to reiterate that the analysis plans are suggestions developed by the working groups for the express purpose of illustrating potential applications of OSE tools and methods to a select sampling of specific TBI care challenges. The analysis plans should not be construed as consensus recommendations of the individual working groups, the workshop participants as a whole, or the National Academies. Working Group A: DevelopMENT OF New TBI Knowledge The phenomenology surrounding TBI, particularly mTBI, is not completely understood. This limited understanding also limits objective diagnosis, the effective management of symptoms at the point of injury, and appropriate acute patient care. In addition, because little is known about the progression or disappearance of mTBI symptoms over long periods of time after exposure to a blast injury, the effectiveness of tri- age, rehabilitation, long-term disease management, and efficient use of community services for mTBI patients are all limited. Thus improving the understanding of TBI injuries and mechanisms is a crucial issue for TBI care providers and managers trying to develop effective treatment protocols. The two specific tasks assigned to Group A both focus on the d Â evelopment of new TBI knowledge. The first involved developing an approach to modeling the neuropathology and clinical effects of blast and concussive injuries on brain functions leading to mild, moderate, and/or severe TBI. The second task was to develop an acute-to-chronic See Appendix G for names of working group members.
Suggestions for Analysis Plans by Working Groups 85 disease model of mTBI showing the evolution of disease states or symp- toms over time for a population of mTBI patients, some who were ini- tially asymptomatic and some who were overtly symptomatic. Because of the need for better data collection on TBI to inform future research, the group also addressed a third issue, the establishment of a national database on brain trauma events in the civilian population. Issue 1: Methods of Measuring Brain Vital Signs Issue 1 focuses on the development of methods of measuring brain vital signs in patients with blast and/or concussive injuries, that is, Âpatients with TBI from blast, blast plus concussion, and concussion alone. The suggested technical approach includes animal and human studies using neuroimaging, neuropsychological testing, and neuropathology assess- ments to (1)Â measure temporal changes in the brain in vivo after a blast and/or concussive event and (2)Â identify biomarkers, brain edema, Âchanges in brain blood flow and volume, and changes in neuroÂchemistry. Once these brain changes and biomarkers have been identified, care providers can determine the efficacy of potential treatments, specific risk factors for the development of TBI, and effective prevention measures. One of the main challenges in treating mTBI is identifying who is affected and how their outcomes develop over time. Therefore, a necessary data requirement for this initiative would be pre-deployment neuropsychological screening to obtain baseline data on a sample of potential patients. The baseline data could then be compared to other pre-deployment and post-deployment tests for changes in brain chem- istry. Data would also be used to evaluate deployable, alternative, and inexpensive methods of measuring brain vital signs, including blood flow, blood volume, and brain edema. A serious problem in post-screening at various points in theater is patientsâ denial of injury and the absence of symptoms. To address this problem, a research station could be established to analyze a group of patients and gather data prospectively at a Level II facility. Imaging research stations would also be beneficial at Level III, IV, and V MHS f Â acilities and Department of Veterans Affairs (VA) facilities. These r Â esearch stations would be capable of performing structural and func- tional magnetic resonance imaging (MRI) to measure brain anatomy. To gather baseline data on asymptomatic individuals exposed to blast for comparison with controls, each vehicle could carry a âblack
86 Systems Engineering to Improve Traumatic Brain Injury CARE boxâ equipped with sensors for characterizing the magnitude and other parameters of a blast. Individual soldiers could also be equipped with black boxes that could characterize the magnitude and specific effects of a blast on that particular soldier. The black boxes would collect neces- sary background information that is not otherwise available. Overall, gathering data on imaging and neuropsychological testing at early stages would make it possible to monitor and compare the effects of an injury event over time and provide a link between vital signs and estimates of event severity. Data are also needed for analyses of systematic gross pathology relative to a patientâs history, including information on previous TBIs, alcohol use, and mental health status. Information for such analyses might be collected from studies currently under way at academic and government research sites. Critical assumptions associated with the analysis plan described above include the capability of MRI at one echelon (at least) in theater; standardization of imaging and support of data collection at Level II through V facilities; and the availability of a technically competent staff. In addition, it is assumed that data will be available on human neuro- pathology, potentially from evaluations of data related to mortality dur- ing the Global War on Terror and that animal studies will accurately mimic the human condition. Critical constraints include obtaining military permission for in-theater research and access to postmortem neuropathological data and establishing a data repository to support collection and long-term analysis. MRI, position emission tomography, and single photon emis- sion computed tomography would also be necessary through Level V at TBI centers in CONUS. Other critical constraints are the execution of research proposals, the relevance of animal models to the human condi- tion, the feasibility of using large animal models to test neuropathology, and the variability of image interpretation among research staff. Performance, outcome, and utility metrics would include (1)Â the validation of animal models via human imaging studies and (2)Â the tracking of patients to ensure adequate throughput of those imaged with mTBI at Level II facilities, as well as those imaged with moderate to severe TBI at Level III through V MHS facilities and VA facilities. It would be necessary for one Level II facility to gather information on a group of patients and a group of control patients, with a minimum of 30 patients per group. Although clinical analyses of MRIs often miss
Suggestions for Analysis Plans by Working Groups 87 cases of mTBI, they still have some clinical utility because they provide data on ventricular volume and atrophy and can identify brain bleed- ing. This information is only available at Level II facilities if they are equipped with MRI imaging capabilities. In addition, neuropsychiatric testing could identify cognitive deficits and could thus be another metric for identifying health outcomes following TBI. Tasks to execute the approach described above would include con- vening a working group of neuropathologists to address questions about TBI and associated exposures and outcomes. Issues to consider include differences between concussive and blast injuries, appropriate measures for these two types of injury, and determining if there is consensus on the sequence of neuropathology after a blast trauma. Tasks would also include establishing a database of results from neuropathological Âstudies and ongoing experiments at government research laboratories and academic institutions. In addition, research proposals would have to be developed to support the proposed animal experiments. Expected outputs include a determination of the differences between mild blast, mild concussive, and mild blast plus concussive injuries as revealed by MRI and neuropsychological measures; the classification of characteristics of mTBI patients who need additional medical surveil- lance and/or treatment; and the identification of temporal characteristics of blast/concussive exposure and longitudinal outcomes. For a single Level II site to implement this analysis plan, Group A estimated that the project would last approximately 18 months and would cost about $3.5 million for in-theater equipment. Data collec- tion would begin four months after initiation of the project and would r Â equire three active-duty personnel and two civilian personnel. Addi- tional support resources (e.g., security) would also be required. Issue 2: Anticipating Downstream Consequences of TBI The objective of Issue 2 is to provide a means of understanding and forecasting the downstream consequences of TBI, including both the later effects of immediate post-trauma treatment and interactions with subsequent TBI events or other psychological traumas. The tech- nical approach to meeting this objective involves modeling a finite state-space stochastic process in which the initial conditions are TBI incidents (timing and conditions), co-morbidity, and treatment. This model would also take into account instances with no co-morbidities
88 Systems Engineering to Improve Traumatic Brain Injury CARE and cases in which no treatment was administered because the injuries were not reported. Necessary data would be collected on anomalies and triggering events and their associated consequences as a basis for developing a list of behaviors and types of triggering events that might be of concern in Â patients with TBI. Longitudinal data from a test sample of long- d Â uration TBI victims (e.g., from the Vietnam Head Injury Study) would also be useful, along with data on short-duration victims (e.g., from Iraq and Afghanistan). In addition, new surveys would be designed and administered to capture the effects of long-duration TBI. Overall, the purpose would be to identify medical problems and behaviors of concern, such as neurodegeneration and adjustment disorder, that are likely to be manifested in 1, 5, 10, 20, or 30 years, as well as triggering events that may complicate TBI and/or accelerate the emergence of those conditions. Data collection would be focused on generating the structure of relevant TBI events to define the states of the system. (The state space would reflect the progression of the TBI patient from acute through chronic stages, and different states would reflect the treatment provided.) The data would redress our current lack of data, and hence our inability to define states and probability distributions of their occupancy times, as well as transitions between states. The initial data collection would involve analyses of patient records and 50 to 100 lengthy interviews for each group, followed by a more formal analysis instrument for samples of 1,000 to 2,000 patients. All data collection would be supplemented with meta-analysis and data mining, with the expectation that the results could suggest appropriate clinical trials of alternative treatments. The overall goal is to establish a baseline frame of reference for detecting changes in the state space. Thus the approach for this analysis plan would involve enumerating the characteristics of a TBI event, co- morbidities, treatment, and baseline characteristics of each victim. Simi- lar documentation of downstream consequences and potential physical and psychological sequelae would also be collected as a basis for making correlations among characteristics of the initial event, downstream con- sequences, and subsequent triggering events. National Naval Medical Center. Ongoing. Vietnam Head Injury Study (VHIS) Phase III. Available online at http://www.bethesda.med.navy.mil/professional/research/vietnam_head_ injury.aspx (accessed September 29, 2008).
Suggestions for Analysis Plans by Working Groups 89 These correlations would require long- and short-term longitudinal histories of TBI patients from Vietnam and Iraq and Afghanistan, as well as appropriate control populations. The Vietnam population could provide an opportunity to examine long-term outcomes not yet realized in the Iraq and Afghanistan population; thus a cohort that was clearly identified as having been exposed to blast injuries in Vietnam would be selected to provide life histories and determine relevant outcomes since their exposures. Data on the forensics of TBI events, including relative location of the blast to the soldier and the type of vehicle the soldier was in when the blast occurred, would also be necessary to define the state at the conclusion of the event. Critical assumptions for this analysis plan include (1) data on TBI victims will be available as a basis for identifying appropriate initial in- terview samples as well as subsequent samples for longitudinal analyses, (2) sufficient patient recall to identify events of interest, and (3) adequate reporting of the salient events. No critical constraints for this analysis are anticipated other than the availability of funding. Metrics include (1) quality of life and its relationship to the events of interest, (2) treatment burden and costs, and (3) downstream needs for patient monitoring and response. Tasks necessary to execute this approach include (1) developing a proposal for the research, (2) secur- ing adequate funding, (3) obtaining an interview sample, (4) execut- ing interviews and preliminary analyses to generate the state space, (5) demonstrating an initial model based on the sample, (6) developing a formal instrument for data collection, and (7) using this instrument to prepare a model for initial application to TBI care. Expected outputs include (1) identification of potential warning signs of downstream consequences of TBI, (2) identification of immediate actions and treatment choices to minimize downstream consequences, (3) classification of a pattern of downstream consequences following a TBI event and its treatment and costs, and (4) comparison of long-term treatment strategies. It is estimated that this project would last four years and require approximately $1 million in funding, with initial operational results expected 18 months from the start-up date of the project. Issue 3: Database on Civilian TBI Events Issue 3 was to create, in cooperation with federal and state agencies and private organizations, a nationwide database of information on the
90 Systems Engineering to Improve Traumatic Brain Injury CARE effects of crashes, explosions, and other traumatic events on the health of the civilian population over time. Data on head injuries from brain trauma events in the civilian sector, many of which may require treat- ment similar to treatment of mTBI on the battlefield, would be made available for analytical and comparative purposes to provide insights into traumatic events on the battlefield. Explosions in chemical plants, natural-gas pipelines, grain silos, car crashes, and other events generate blasts that have many characteristics in common with munitions explo- sions. There are also important differences that may be significant in recognizing the effects of battlefield injuries. The national database would be continuously updated and made available for analysis by all interested parties. Data on the nature of brain trauma events, the surrounding conditions, the effects on personnel, and medical diagnoses of immediate and long-term effects would be required for implementation of this approach. A critical assumption is that data from brain trauma events in the civilian sector would provide information important to improving the un- derstanding of TBI effects in the military population. A critical constraint is that there is no central federal or civilian agency that could collect and archive data of this nature, let alone data detailed enough for analysis. Performance, outcome, and utility metrics include identification of the characteristics of brain trauma events, primary clinical effects on humans, primary effects on brain structures and their impacts on humans, and long-term clinical effects on humans. To execute this ap- proach, a federal program would have to be identified that would be responsible for receiving and archiving data, and a mandate or incentive system would have to be implemented to ensure that all traumatic brain events in the civilian sector were promptly reported to this agency. The expected output would be a large civilian database that MHS could use to augment its analysis of TBI and make comparisons with data col- lected in theater. Implementation of this approach would require identifying one or more advocates for the creation of the proposed database, as well as the development of an organizational structure to support the activity. Although the costs cannot be estimated at this time, this would be an ongoing effort, and resources to accomplish it would have to be carefully explored. The U.S. Department of Transportation, which collects data on vehicle crashes, and perhaps some other agencies or organizations, could provide some support.
Suggestions for Analysis Plans by Working Groups 91 Working Group B: Detection and Screening OF TBI Conditions MHS needs better testing (cognitive tests, brain scans, etc.) for TBI, particularly for mTBI. The test results and other information could be used to develop an effective, efficient screening process that takes into account Type I (sensitivity) and Type II (specificity) detection errors. In the case of mTBI, physical symptoms may not signal the extent of the injury. Moreover, the current screening system relies on self-reporting of TBI causative events or symptoms in theater or at the end of a de- ployment. This system has been less than effective because individuals have multiple disincentives for self-reporting both during and after a deployment. The costs of false positives and false negatives are significant. mTBI âfalse alarmsâ remove healthy soldiers from service at significant cost to the military mission. However, if undetected, mTBI can impair job per- formance during a deployment, putting the individual, other soldiers, and the mission at increased risk. If not detected and treated promptly, an initial mTBI increases a soldierâs risk of subsequent TBI and/or may result in long-lasting symptoms post-deployment that will degrade his or her quality of life. Unfortunately, little objective information is available about the onset and progression of mTBI that can be used to assist in detecting and screening for mTBI injuries. There is, however, a good deal of sub- jective information (e.g., medical experience; neurological, cognitive, and psychological testing; imaging; questionnaires), as well as data on the incidence of mTBI, that could conceivably be used as an interim diagnostic vehicle for assessment, detection, and screening programs and in making âreturn to dutyâ (RTD) decisions. Working Group B focused on two issues: (1)Â the development of systems engineering models for evaluating and improving the current TBI screening process, particularly for mTBI; and (2)Â the development of a predictive diagnostic model for mTBI (i.e., a means of Âestimating See Appendix G for names of working group members. Disincentives are based on the stigma associated with the injury; soldiersâ reluctance to risk being removed from their units; leaving their comrades in arms short-handed for an invisible or (mis)perceived âminorâ injury; soldiersâ desire at the conclusion of a deployment to âjust get homeâ and not prolong the post-deployment evaluation; and soldiersâ fears of the negative implications of a positive diagnosis for long-term military careers.
92 Systems Engineering to Improve Traumatic Brain Injury CARE the probability that an individual soldier returning from the field [or remaining in the field] has mTBI). The group focused mostly on developing an analysis plan to advance the first objective, which ap- pears to be the more difficult technical (i.e., modeling) challenge. Less consideration was given to the second objective, which would be easier to model. Issue 1: Evaluate and Improve the Current TBI Screening Process In the first analysis plan developed by the working group, the expected outputs would be an improved understanding of how well the current screening and diagnostic processes work and mathemati- cal models that suggest opportunities and requirements for improving them. Specifically, the models developed in the course of the proposed study would (1)Â recommend a staged approach to the screening instru- ments that should be used, specifying when, in what sequence, and in what locations (e.g., in-field, on the base, post deployment) they should be used, based on estimated risks, sensitivities, specificities, and costs; (2)Â evaluate the costs and benefits of the new screening tool, technology, or approach; and (3)Â assess usability and compliance issues associated with the screening and diagnostic process and the benefits of improving upon these. The working group identified five complementary technical ap- proaches: (1)Â determine problems or shortcomings of the current approach; (2)Â identify other approaches; (3)Â develop descriptive models of the current process; (4)Â develop prescriptive models for optimizing the screening process; and (5)Â develop learning models through data- mining techniques. Whatever models are developed, they must be able to accommodate and/or investigate data uncertainties (e.g., unknown costs, sensitivities, compliance). One approach to determining the limitations of current processes would be to use mapping based on interviews, observations, focus groups, and surveys. Once these data are obtained, usability testing could be conducted, as well as analyses of cognitive tasks and cognitive work and analyses of training and implementation requirements. Another approach would be to identify alternatives to the current approach, either by designing and evaluating (rapid prototyping) bet- ter methods or tests for screening or by looking to other domains for
Suggestions for Analysis Plans by Working Groups 93 ideas. For example, investigators could study how athletes, miners, and industry workers at risk of TBI are tested. A third approach would be to quickly develop descriptive models to determine how the current processes perform. These models could include probability models formulated as a Markov process, Monte Carlo models, cognitive-task analyses, judgment models (to identify the key cues used by experts), resource-requirements models (to identify the screening load), and throughput models (see Appendix H). In addition, prescriptive models could be developed to optimize the screening process. These could draw upon classic optimization models (such as those described in the following paragraph), Markov decision processes (MDPs), partially observable MDPs (which lend themselves to handling uncertainties or partially observed data), and simulation mod- els. Based on prescriptive models, one could predict the performance (e.g., cost, number of true and false positives, number of true and false negatives) of a given screening process and then adjust details of the process to improve performance. Another conventional approach to addressing uncertainties would be to conduct parametric analyses and sensitivity analyses on models, for example, to determine which of the data elements initially targeted for collection had the strongest effect on the model results and, there- fore, require the most improvement, perhaps by further data collection, analytical studies, and so on. Finally, signal-detection theory models could be used for making a diagnosis in a situation in which one must distinguish between signal and noise and determine an optimal criterion for classifying an individual as positive or negative for mTBI. The group identified several objectives for prescriptive optimization models. First, a model could be developed to minimize the total costs of the current screening process, including both the direct costs of screen- ing and the costs associated with the potential harm from false positives and false negatives. Another objective could be to minimize the expected number of tests and screens administered. A third objective might be to minimize the number of misses while controlling for the number of false alarms. Classically, one could minimize total costs by including known d Â irect costs, the costs of false negatives, and the costs of false positives and then conducting a sensitivity analysis. However, many of these costs cannot be clearly identified or quantified. Another way to model problems such as these is to remove the costs from the equation entirely
94 Systems Engineering to Improve Traumatic Brain Injury CARE and aim for a certain performance level with minimal screening costs. Because all of these objectives are likely to be important, however, one could formulate models that assess multiple objectives or criteria and then try to optimize the process with respect to multiple competing criteria. A multi-criteria optimization model, for example, might Âenable one to assess trade-offs between minimizing total costs, minimizing harm, and minimizing false negatives. Note that the same general modeling approaches could be used for deployment and post-deployment screening scenarios. Issue 2: Develop Predictive Diagnostic Models for mTBI The second issue discussed by the working group was the develop- ment of predictive diagnostic models, a means of estimating the prob- ability that an individual soldier returning from the field will have mTBI. Similar to the multiple-model approach described above, the proposed technical approach is to predict the probability of mTBI using several modeling approaches and to compare these approaches to determine which ones are most effective. Suggested approaches include basic prob- ability models, logistic regression models (which assume good data on an individualâs deployment history and TBI status), Bayesian networks, influence diagrams, and fuzzy logic models (the latter three can work with imprecise information; indeed, fuzzy logic models have already been used for predictive modeling of TBI in civilian populations) (Guler et al., 2008) (see Appendix H). Each of these approaches is fairly straightforward. The model developer would choose the one most likely to be useful based on process particulars and currently available data, following the general development approach described below. The modeling approaches listed above should be treated as comple- mentary and iterative. Given our limited understanding of the sensitiv- ity and specificity of the current screening process, one could begin by developing a simple probability model of the current process and use it to conduct âwhat if â analyses to get a better understanding of how well the system performs. Later, the model could be expanded into formal optimization frameworks, but typically a great deal can be learned about how to improve outcomes by using the model to conduct simple trial- and-error inquiries. In other words, one would develop a model, validate it, start to use it to identify improvements, and then build an optimization structure
Suggestions for Analysis Plans by Working Groups 95 around it. Because the first model is often not exactly on target (either because it does not capture important aspects of the problem or because it requires unobtainable data), successful modeling almost always fol- lows an iterative process. In this context, multiple types of models are often simultaneously prototyped to get an idea of which ones will be most useful. Usually no single model is chosen, but a few models are used together in complementary ways to address or illuminate different aspects of the problem. Following this strategy, one approach would be to fund exploratory work, through academic grants, consulting, and/or seed funding for a number of short-term pilot studies (approximately 10 projects of 6 to 12 months duration), possibly focused on a subpopulation. Funding recipients would spend the first 6 to 12 months developing an in-depth understanding of problem specifics and model requirements, undertake preliminary data collection or knowledge elicitation, and build and evaluate preliminary models. If seed funding were provided, successful prototype tests could lead to further funding for longer term studies (one to three years) to refine and expand the most promising approaches. This might be followed by a one- or two-year rollout on a larger scale. Data requirements for the proposed research would include po- tential input variables, output performance data for model validation, and data for evaluating the effectiveness of system redesigns. Screening m Â odels would require data on different kinds of screening tests and methods, their costs and risks, their false negative and positive rates, the costs of false negatives and false positives, transition probabilities, compliance rates, resource requirements, and the time required to run a test and reach a decision. Data would also be gathered on the overall screening process so that when different screening tests are integrated into a more complete screening process, the effectiveness of the whole process can be measured. Data for model validation include output data, such as unit effectiveness (e.g., readiness, morale, cohesiveness in action) and other militarily relevant information. For diagnostic models, data needs may include predispositions (e.g., genetics, physical and mental factors), past incidences (e.g., num- ber, frequency, severity, consequences), and exposure (e.g., role in the armed forces). Further data requirements will constitute a much longer list, some of which would be identified in the development of the pilot models described above. Other information needs might include policy
96 Systems Engineering to Improve Traumatic Brain Injury CARE requirements or requirements based on a more in-depth understanding of problems with the current screening process, such as, for example, the Post-Deployment Health Assessment and the Post-Deployment Health Reassessment. In general, several pre-test/post-test comparison measures would be useful inputs to either type of model. Many types of data might be collected about the population to help clinicians infer whether mTBI is present in an individual, such as who patients are, where they have been, current and past diagnoses and treatments, baseline mental states, and so on. Evaluation metrics might include RTD, unit effectiveness, suicide rates, total costs, and the effectiveness of screening for the detection of TBI and related problems and the ability to distinguish between them. Critical assumptions for success are that the data necessary for build- ing models are available and that policies can be changed if they interfere with optimal decision making. Critical constraints include funding, the availability of non-financial resources, and competing priorities. Other constraints include the need to train people, the deployment of the screening process, tracking requirements and compliance, and build- ing and maintaining the data infrastructure to support the improved screening process. Working Group C: Coordination and Communication FOR TBI CARE The primary charge of Working Group C was to develop the struc- ture of a TBI information system to track, monitor, and cue care delivery for all TBI patients, no matter what the severity of their injuries. The Enhanced Post-Deployment Health Assessment (PDHA) Process (DD Form 2796). DOD Deployment Health Clinical Center, Walter Reed Army Medical Center, Washington, DC. Available online at http://www.pdhealth.mil/dcs/DD_form_2796.asp (accessed September 29, 2008); Post-Deployment Health Reassessment (PDHRA) Program (DD Form 2900). DOD Deployment Health Clinical Center, Walter Reed Army Medical Center, Washing- ton, DC. Available online at http://www.pdhealth.mil/dcs/pdhra.asp (accessed September 29, 2008). These requirements and constraints were drawn directly from the DVBIC Working Group on the Acute Management of Mild Traumatic Brain Injury in Military Operational Settings, Clinical Practice Guideline and Recommendations, dated December 22, 2006, which lists many key elements only some of which are called out here. See Appendix G for the names of members of all working groups.
Suggestions for Analysis Plans by Working Groups 97 system would be useful for clinical monitoring and follow-up and would be accessible to, and would cue, all patients, patient families, and other relevant providers in the MHS, VA, and civilian sector. In addition, the group was asked to develop a methodology for the coordinated delivery of services for TBI and related co-morbidities immediately following trauma exposure. The methodology was required to take into account the needs and preferences of the patient and family members, as well as the resources (e.g., number and type of providers available) and infra- structure of the relevant health care system. The objective of the analysis plan is to address DODâs lack of a system-wide approach for tracking and monitoring TBI patients to e Â nsure the effective management of their complete care. Problems in the coordination of care have arisen between the MHS and VA and among all care providers at different levels of care and at different medical facili- ties (Cope et al., 2005; Sayer, 2006). The analysis plan and methodol- ogy described by the working group would also improve the timeliness, coordination, and efficiency with which TBI care resources (e.g., care proÂviders, equipment, materiel, supporting organizations, and infrastruc- ture) are brought together to address the needs of TBI patients during the first two or three days after the initial injury. Improving the operational responsiveness and coordination of the TBI care system will also improve patient outcomes and the efficiency of using scarce resources. The desired output of implementing the analysis plan would be a proactive information system that facilitates the tracking, monitoring, cueing, coordination, communication, and scheduling of care for TBI patients from âcradle to cure.â Such a system would also ensure that in- formation flows and flows of care are aligned to provide effective, timely status awareness and response capability for TBI patients. An additional output would be an operational model and/or process methodology that could be used for the real-time allocation of TBI resources, thus ensur- ing that the delivery of clinical services is coordinated and responsive to patientsâ needs. In working its way through the development of an analysis plan, the groupâs initial discussion focused on key considerations, such as that many military personnel who experience mTBI are not identified and, therefore, are not tracked or treated. Sometimes the manifestation of TBI symptoms is delayed. Sometimes, even if detected, TBI symptoms are confused with other medical conditions or are ignored by the soldier or his or her unit because of the urgency and gravity of the military
98 Systems Engineering to Improve Traumatic Brain Injury CARE mission. Moderate and severe TBI are easier to identify and to differen- tiate from other medical conditions. However, even severe TBI may be initially overlooked in the presence of other traumatic injuries. Another general consideration is that the continuum of TBI care involves moving the patient across time, across functions of care (e.g., re- suscitation, acute care, rehabilitation, disability evaluation, community/ unit reintegration, chronic management), across geographic locations, providers, and treatment institutions of the Army, Navy, Air Force, VA, and private medical facilities. The continuum of care requires patient movement, caregiving, data recording, use of medical equipment, and medical supplies, as well as interactive communication among the pa- tient, professional care providers, family members, social networks, and communities. Treatment or acute management of moderate or severe TBI is not as problematic as the long-term management of severe cases or the detection, tracking, and management of mTBI cases. The objective of the analysis plan is to improve communication and coordination of care so that all TBI patients get the right care at the right time in the right place by the right provider. In an ideal system, patients, their families, and their care providers have access to necessary information and are fully informed and empowered to improve and ac- celerate recovery. The plan developed by the group provides a framework for achieving this objective. The technical approach necessitates detailed mapping of the TBI care process showing the flow of patients through events (nodes), the time between nodes, and the resource requirements, including prÂoviders, information, and medical logistics (such as assessment tools and pro- tocols) at each node. The care process is different for unrecognized mTBI cases, recognized mTBI cases, and moderate or severe TBI cases. P Â rocess maps would provide a basis for DOD to construct models to test whether changing events, changing the time required to reach a node, and/or changing the resources would improve the efficiency of care. Information relevant to TBI needs that is currently being collected at various care levels in various medical facilities in the United States could be linked using current and evolving information technology (IT) integration techniques. DOD could create a virtual knowledge- m Â anagement infrastructure for enterprise-level, institution-level, clinic-level, and patient-level knowledge and decision making. The data i Ânfrastructure could draw together traditional individual and aggregate data on medical care, as well as video recordings (patient interviews in
Suggestions for Analysis Plans by Working Groups 99 clinical settings or remotely from home), and other archived data that could be used retrospectively to assess TBI exposure (black box video from military vehicle, personal, and environmental sensors). Another technical approach would be networking technologies using Web-based systems for patient-to-provider, family-to-provider, provider-to-Âprovider, and patient/family-to-patient/family communication. The structure of the integrated system is highly dependent on the intended uses of the data or information it provides. To begin designing a structure, one must first identify all of the databases that would be linked. The level of detail would be specified in the design depending on how the data were used. For example, a field medic assessing a soldier for TBI following the explosion of an improvised explosive device (IED) would have different data inputs and outputs than a neurosurgeon in the United States involved in comprehensive acute clinical manage- ment. At the enterprise level, data formats may be less specific and more a Â ggregated than data at the medical-facility level used for the manage- ment of individual patients. The structure would have portals for access by multiple users, such as TBI patients, families, providers, MHS managers, and the general public. It would also have to be accessible from remote locations via portable wireless devices. One glaring deficiency in the current data structure is the absence of a uniform taxonomy for TBI, especially mTBI, in internationally used medical diagnostic database systems, such as the International Classification of Disease (ICD). In addition, metadata requirements would be integral to the structure to provide a context and organization for the data. Integrating multiple databases would require core (master) identi- fiers common to the component databases. The data outputs would depend on the intended use of the information. The data requirements would not be limited to numbers and text, but would include data from video, voice, sensors, and other dynamic behavior, that have been compressed and archived and could be used âon call.â Nor would data be limited to existing databases. New data would be generated, stored, and linked within the integrated system; examples might include a database that measures progress or outcomes when us- ing new clinical protocols or a database that demonstrates whole-system performance using specified metrics. A basic assumption for the information system is that different organizational entities involved in TBI care, such as military treatment
100 Systems Engineering to Improve Traumatic Brain Injury CARE facilities (MTFs), VA MTFs, and TRICARE-sponsored private facili- ties, would share data they had collected on behalf of their patients and on behalf of the accountable health system. Another assumption is that effective communication and coordination would entail much more than the application of information technologies. It would involve the commitment of resources, the establishment of compatible policies and procedures, and synchronous planning at the tactical (bedside), opera- tional (hospital), and strategic (enterprise) levels involving all affected stakeholders. A final assumption related to information technologies is that existing database systems can be identified and integrated. Any plan designed to improve communication and coordination in the comprehensive delivery of TBI care for military personnel would be subject to major constraints. Cultural differences among the Army, Navy, Marines, Air Force, VA, and the private health care systems, reflecting mainly differences in tradition and mission, are barriers to transparent interoperability. Even at the data level, there are numerous constraints. Data security imperatives and patient privacy concerns can limit, delay, or prohibit data sharing. Databases are often fragmented (e.g., the military services may have similar, but not identical, databases) and decentralized, making integration difficult. Data collection in the field, especially in a combat zone, is very challenging and is always sub- ordinate to the military mission and operational safety. The capabilities of frontline combat military service members may be limited, especially if the data collection task is time-consuming or complex. One of the main constraints on TBI data is that there is no universal diagnostic code for TBI. Instead, related codes are used, such as skull fracture [ICD-9 800-804; ICD-10 SO2], concussion [ICD-9 850-850; ICD-10 SO6], and late-effect codes. In addition, an incident that may cause an injury, such as an IED explosion 50 yards away that knocks down and dazes a soldier, may be difficult to characterize or document. Some- times the exposure has delayed effects, and sometimes it goes unnoticed or unreported because no symptoms persist. Thus these data may not be captured. Even if symptoms persist, they may not be reported if the service member believes that reporting them will delay his or her return home. Metrics must be relevant to all key stakeholders. For patients, met- rics include ease of access, utility (value), and timeliness of the informa- tion they need to facilitate their recovery. For family members, metrics would be based on whether they have access to information about their son or daughterâs condition, treatment plans (with appropriate consent),
Suggestions for Analysis Plans by Working Groups 101 and available resources and whether they could communicate critical in- formation to the provider team in a timely, effective way. For providers, metrics would focus on whether they could access critical information about their patients, including the results of diagnostic tests, laboratory tests, and records of treatments and medications. Provider teams also need access to patients and their families in some longitudinal manner and access to the most current policies, protocols, procedures, and devices for optimal TBI care. For the enterprise manager, metrics would measure access to data generated from treatment sources so they could determine the number of cases by severity level, incidence trends, costs, and outcomes of TBI care. Also the MHS (enterprise) man- ager needs to know the quality and timeliness of the data inputs. Execution of this plan would require eight major tasks. The first, fundamental step is mapping the care process for TBI to identify and organize time, place, person, data requirements, and decision points. Second, available databases would be indexed (located and character- ized) to ensure that current data from multiple sources are properly structured and included. Third, strategies and tools would be identified for integrating databases (building an architecture). Once databases have been identified and characterized, information technology tools could be used to link them. A longer term goal would be to establish and link new databases. The analytical knowledge would be included in the data infrastructure to develop, distribute, and use information to inform future health care policy related to TBI. The fourth task is to develop an interactive communication system, an open architecture system that can accommodate and connect the informational needs of patients, providers, families (caregivers), and the community using multiple paths, multiple modes, and multiple devices in every geographic space where patients reside. This task requires le- veraging the Internet using Web 2.0 or further evolved systems (includ- ing Semantic Web services), a common portal that allows wireless and remote access, provides security protection, and has interactive capabili- ties. Assigning a lifetime e-mail address to each service member could facilitate longitudinal two-way communication and outreach. The fifth task is to establish mechanisms for clearly documenting and tracking each TBI case, which may be especially challenging for mTBI cases. Developing uniform identifiers for TBI is essential, especially for mTBI. When available, archived video collected in Â vehicles and/or in high-traffic areas in the combat zone could be used to identify suspected
102 Systems Engineering to Improve Traumatic Brain Injury CARE cases of TBI early. This task also involves identifying a mechanism for col- lecting and archiving real-time and non-real-time multimedia data. Exist- ing and novel telemedicine tools could be used to monitor and provide care for longer term TBI cases, especially in remote locations. Sixth, a registry for TBI cases (somewhat like a tumor registry) would be established for long-term follow-up of individuals and the evaluation of population trends. Seventh, GPS technology would be used to identify the location of service members with TBI and to create density maps to inform decisions about the location of care and commu- nication resources. Finally, a small portable device would be developed that can store and retrieve the medical information of service members (essentially, a hardened individual electronic health record). Data would be backed up wirelessly at the time of entry to prevent data loss in case the device is damaged or lost. Major output elements to this plan would be a TBI-care process map; an integrated data system with an analytical knowledge center; a dedicated communication portal to meet the information needs of patients, providers, families (caregivers), and the community (public); and a TBI registry. The major output of the analysis plan would be that the military service medical departments, VA, and TRICARE system leaders would develop and refine plans to improve the compatibility of care practices and data systems for TBI care. The director of the new Center for Psy- chological Health and Traumatic Brain Injury could play a leading role in implementing this plan. Clinical expertise could be drawn from the DVBIC staff, and engineering expertise would be available from sources internal to and external to DOD to facilitate process mapping and to provide advice on information technologies and OSE. Initial efforts would focus on feasibility analyses, cost and time estimates, and plans for pilot testing elements of the plan. Working Group D: MEASURING AND FORECASTING THE DEMAND FOR TBI Care The impetus for addressing the topic of demand for TBI care was to provide policy makers with data to make informed decisions about the See Appendix G for names of working group members.
Suggestions for Analysis Plans by Working Groups 103 resource requirements for meeting the future medical needs of service members and veterans with mTBI and to evaluate the cost effective- ness of TBI prevention and mitigation. Effective management of the resourcesâproviders, facilities, equipment, and so onârequired for the delivery of TBI care will necessitate an understanding of the current and projected need for care. Three specific analysis objectives were assigned to Working Group D. First, based on historical data on mTBI, develop a statistical estimate of the number of mTBI cases in the population of military personnel who participated in OIF/OEF; second, develop a methodology for forecasting future mTBI cases in the military population, including new and previously undiagnosed cases; and third, develop the elements of, and a process for assessing the value of measures to prevent the occur- rence of TBIs. These three objectives are highly interrelated. The number of m Â TBIs among military personnel is not known with fidelity because it is generally accepted that there are unrecognized cases in the popula- tion. There are several possible reasons for this. First, the injury may be relatively subtle (changes in the ability to concentrate or changes in mood, for example) and therefore difficult to appreciate or acknowledge immediately. Second, injured persons may not report symptoms because they attribute them to other causes or because they wish to continue performing their duties. Third, symptoms may not develop immedi- ately, or other injuries may mask mTBI-related deficits. Thus estimating the number of cases now and in the future is likely to require the same analysis approach and data. For this reason, the working group decided to address these two objectives together. The initial challenge identified by the group was terminology. Data- bases, service members, medical providers, and other potential sources of information do not all use the same terms to describe mTBI injuries; in some cases the terms are used by more than one source but are defined differently. When the goal is to combine data to generate estimates, these differences create obvious problems. The group therefore assumed that a consistent terminology could be imposed on the data, that sufficient TBI data are available to make informed estimates, and that TBI diag- noses in the electronic medical records are accurate. To achieve the analysis objectives, the group proposed that the military population be nominally characterized by branch of ser- vice, location in theater, and exposure risk. The input variables to the
104 Systems Engineering to Improve Traumatic Brain Injury CARE modelâand thus the data required to perform analysesâcould include the following: â¢ the number of mTBI cases diagnosed in OIF/OEF personnel at DOD and VA facilities and by other service providers â¢ the reported prevalence of TBI as a function of total casualties â¢ the distribution of reported TBI diagnoses by injury mechanism and severity â¢ the number of deployed service members â¢ the distribution of personnel by service and component10 â¢ the distribution of personnel by theater (OIF versus OEF) and, in each theater, by combat or non-combat primary assignment â¢ the number of deployments11 and duration of deployments12 â¢ the rate of use of IEDs and other concussive devices by theater and region, if available It was understood that these values would vary over time and that a determination would have to be made on the most appropriate timeÂ scales to use in constructing the models. The data would be derived from combat, medical, and government records, as well as historical data on OIF/OEF. Thus the models would combine available data on diagnosed m Â TBIs with information on exposures and risks known to be associated with mTBIs to generate an estimated prevalence of TBIs13 by mecha- nism (blast, assault, fall, vehicle accident, penetration wound, etc.) type14 (mild, moderate, or severe), and time after the injury-producing 10 Active-duty and reserve components (comprising both the âReserveâ and the National Guard) have different levels of training and may be assigned different missions, which in turn may affect potential exposures to TBI events. 11 That is, is this the first time this unit has been there? the third time? 12 That is, how long has the unit been there? Is this the first week? the last week of 15 months? 13 Prevalence is defined as the total (diagnosed plus unrecognized or unreported) number of cases in a given population at a particular point in time. 14 TBIs can also be classified by the cause of the injuryâclosed-head trauma versus penetrating wound. âPenetratingâ is sometimes listed as a fourth TBI category, along with mild, moderate, and severe, because of the implications of this form of wound to the care of the patient. A workshop participant pointed out that phenomenology may also be importantâwhether an mTBI case presents as a headache, behavioral changes, and so on.
Suggestions for Analysis Plans by Working Groups 105 event.15 These could be calculated as probability distributions to yield an understanding of the uncertainties associated with the estimates. The prevalence of mTBI, the ultimate goal of the exercise, will necessarily generate the broadest distributions, because the number of mTBIs is much less certain than the number of moderate and severe TBI cases. The most critical constraint on constructing and implementing the models is the ability to develop a âgood-enoughâ understanding of the progress of cases, from unrecognized to diagnosed, to be able to evalu- ate how well the surrogate exposure measures delineated above predict outcomes. A second constraint is the quality (availability and usability) of the data. Workshop participants indicated that the problems include the difficulty of identifying all relevant databases,16 some of which may be Â service-specific or may be formatted in ways that are incompatible with other databases (e.g., coded in incongruent software applications or the same label used for different data). They noted, however, that efforts are under way to make databases more compatible and to create a central registry, although these efforts are still in the early stages. Data-mining techniques could be used in the short term to overcome some of these difficulties. The value of the first (current population) model is that it will yield a more accurate picture of the impact of TBI on the military mission and its burden on the MHS and the VA health and benefits systems. The first model will also feed into the second (forecast) model, which can be used not only to project future demands on health and benefits systems, but also to predict the possible effect of various protective measures. These may include advancements in technology (e.g., more protective helmets), procedural changes (e.g., modifications to the as- sessment protocol), and new medical treatments (e.g., drugs that limit brain damage after injury). An understanding of how different injury mechanisms affect overall prevalence may also lead to more effective interventions and more efficient allocations of resources for the preven- tion of TBI injuries. The working group identified several unresolved questions associ- ated with the two prevalence models: 15 Was the injury diagnosed at the time of the event? . . . post deployment? . . . after the person left the military? 16 Relevant databases may include the Armed Forces Health Longitudinal Technology Application, Clinical Data Repository, TRANSCOM Regulating and Command & Control Evacuation System, and others.
106 Systems Engineering to Improve Traumatic Brain Injury CARE â¢ whether particular manifestations of mTBI are more likely to re- solve quickly or can be successfully managed with relatively low- intensity treatment (e.g., short-term removal from the battlefield environment) â¢ the probability and determinants of delayed or sustained symp- toms from a TBI â¢ the probability and determinants of long-term sequelae from a TBI â¢ the cumulative effect of multiple TBIs â¢ the extent to which information about TBI from other populationsâexplosive ordnance disposal personnel,17 miners and building demolition workers,18 and other combatants in other militariesâis relevant to the U.S. military experience in a combat environment Answers to any of these questions could be incorporated into the mod- els to refine the estimates. The third objectiveâto develop elements of and a process for assessÂ ing the value of preventing TBIâis necessarily more complex than the first two, because prevention and injury mitigation can mean prevent- ing TBI-causing events and/or providing better protective measures for mitigating injuries or keeping the health status of injured personnel from deteriorating by using best practices and new medical and other therapeutic technologies. Furthermore, assessing the value of prevention would entail not only the costs saved in lost-duty time, in impacts on other co-morbidities, in possible administrative separation, and in not having to provide health and social support services to the service member/veteran and family, but also changes in the injured personâs quality of life, a criti- cally important concept that is extremely difficult to quantify. The intent behind the third objective was to develop a means of evaluating alternative TBI protection initiatives and, generally, to compare the utility of prevention and treatment approaches. Given 17 The Defense Advanced Research Projects Agency (DARPA) is currently investigating this at the Marine Corps Air Station, Cherry Point, N.C., Explosive Ordnance Disposal School. 18 Dr. Mouratidis, a member of the group, observed that one has to be careful about generalizing from other blast exposures, as her research indicates that exposure to a single IED blast may have a far greater effect than exposure to a large number of controlled blasts. COL Poropatich, also in the group, pointed out that one consideration is whether an individual is exposed to blast alone or blast plus the subsequent shockwave.
Suggestions for Analysis Plans by Working Groups 107 a Â ssessments of the various types and levels of prevention and injury miti- gation, and using the previously discussed prevalence (and incidence) estimation models, it could be possible to estimate the new incidence of TBI by category (severity, mechanism, theater, etc.) as a result of the prevention, reduction, and/or amelioration of casualties attributable to the implementation of new technological and/or system protective initiatives. These new estimates of incidence and prevalence could then be used in the models of other task forces to determine the short- and long-term reductions in costs, resources, facilities, and locations of treatment for patients and their families in the DOD, VA, and civilian health care systems. Working Group E: Capacity, Organization, and Resource Allocation FOR A TBI Care System19 TBI care in the MHS involves complex interactions at the tactical and strategic levels. In addition, numerous co-morbidities are associated with TBI, including mental health conditions and physical injuries. Addressing these issues entails an analysis of TBI-related capacity issues, organizational issues, and associated resource allocations within the MHS. Capacity issues involve requirements for providers, facilities, and equipment; organizational issues focus on assessing the cost effective- ness of TBI care, evaluating changes to the care system, and analyzing the impacts of multiple TBI care systems with different organizational structures. In addressing these issues, it is important to take into account system-wide interactions, as well as relevant TBI co-morbidities from a âsystemsâ or âenterpriseâ perspective. Group E was asked to develop a description of elements, processes, and activities that represent the dynamics of a complete episode of TBI care for injuries at all levels of severity; the description must include demand for TBI care, care processes (protocols), and resources. The purpose of the description is to inform the design of an approach to the development of a stand-alone model of the TBI care system or to enrich an existing enterprise-level health care delivery model that would include TBI system elements, care processes, and resources. The overall output was envisioned to be a âTBI systemâ model or âenterprise-levelâ health care delivery model with the potential to address 19 See Appendix G for names of working group members.
108 Systems Engineering to Improve Traumatic Brain Injury CARE a broad spectrum of TBI capacity, organizational, and resource-allocation issues. The expectation is that, if the model(s) are properly structured, it might point the way to the prospective design of a TBI system of care. However, much of the work of Group E was predicated on the output of models or analysis plans developed by Groups A through D. The second task of Group E was to outline the structure of a r Â esource-allocation model or methodology for allocating scarce TBI care providers to meet the demands in theater and in CONUS for all TBI cases, no matter what the severity of the injuries. As an alternative to this approach, group members were asked to consider a system of assigning TBI patients to care providers. MHS care providers with specialized knowledge of TBI are often responsible for treating other diseases and are not geographically distributed in a way that enables them to provide efficient, effective care to existing and future TBI patients. Thus MHS needs a method for determining the best use of these scarce resources in the short term. The purpose of an analysis of the resource-allocation issue was to address the shortage of MHS care providers with expertise in TBI treatment and management by providing a methodology for allocating scarce TBI-capable care providers to meet the needs of TBI patients in theater and in CONUS and to identify high-priority requirements for additional TBI care providers. To achieve the overarching objective for Group E of developing a model for the allocation of resources in the MHS TBI care system, several important factors had to be consideredâthe first of which was deciding on a particular TBI model to follow. Since approximately 90 percent of TBIs experienced in theater are mildâand because moderate and severe TBI are both well understood and appropriately treatedââthe group decided the model for this analysis plan would be treatment of mTBI. The questions to be addressed were: which TBI enterprise perfor- mance metrics matter and what affects them; which processes, resources, and organizations are necessary; which elements of the system are criti- cal to success; and how scarce resources should be allocated. Important performance metrics included (1)Â coverage of care, (2)Â the percentage of mTBIs detected and treated appropriately, (3)Â outcomes in terms of patient safety and successful return to duty, (4)Â the percentage of individuals able to return to work, and (5)Â quality of life after an mTBI. Costs and trade-offs in the TBI care system were also considered.
Suggestions for Analysis Plans by Working Groups 109 To determine the necessary processes for the model, the group as- sessed how these processes are impacted by co-morbidities across the continuum of care, from disease prevention to patient/soldier rehabili- tation and reintegration into society. The phenomena evaluated ranged from the physical to the social and included the need for a comprehen- sive monitoring process focused on awareness and education. The issues addressed included the typical sequence of events leading to the develop- ment of mTBI and differences between the presentation of symptoms and the observation of these symptoms over time. The consequences of the passage of time represent a large gap in our understanding of the de- velopment of mTBI; it is necessary that we understand when particular events associated with mTBI occur and whether the disease progresses if no treatment is administered. Resource allocation would also be assessed for all levels of TBI severity, and process measures related to outcome metrics would be analyzed. Process characterizations in this analysis plan range from prevention to reintegration and include screening, diagnosis, treatment, and reha- bilitation. Specific areas of focus were physical, cognitive, emotional, behavioral, and social factors, with monitoring of each to determine changes in an individual as a result of mTBI, the causes of the changes, how changes should be treated to maximize function, and how the individual can be helped to re-engage in work and social activities. Monitoring these factors requires awareness, education, pre-mission operational checks, subjective reports, physical assessments, and change management. Critical issues for the analysis include the sequence of events be- fore and after the occurrence of TBI, the presentation of symptoms and whether they are affected by the severity of the injury, observation of symptoms and subsequent effects of co-morbidities, and patient e Â ntrance into MHS. Other phenomena to be addressed by the model were (1)Â patientÂ experiences compared with the experiences of others who are with the patient (family members, professional care providers) and (2)Â queues of individuals with progressing symptoms, including (3)Â the consequences of delayed treatment. The most salient difference between TBI and other diseases identified during the assessment were the difÂficulty of detecting mTBI, which requires treatment of symptoms that cross physical, cognitive, emotional, and psychological boundaries, rather than the fundamental causes of mTBI, which are poorly understood at present. Consequently, mTBI care requires multidisciplinary, Âcoordinated
110 Systems Engineering to Improve Traumatic Brain Injury CARE care (more than 48 care programs have been developed). In the MHS this requires that each patient have a case manager (not a care provider) who serves as care coordinator. There is no evidence-based treatment for mTBI that accounts for multiple symptoms and co-morbidities. Thus an assessment of this care system must include analogues in the management of other complex diseases, such as diabetes and cancer. To manage differences between mTBI and other diseases, variabili- ties in case management in MHS and VA at different locations must be considered. At present, there is one track for mTBI identified in theater, another for other injuries occurring in conjunction with mTBI, and a third track for mTBI recognized only after a period of time following the incident. Although the first point of contact is responsible for a patient regardless of the evolution of the patientâs care, all patient care is ultimately the responsibility of specialists regardless of whether they have directly interacted with the patients. The technical approach to this analysis plan involves the develop- ment of a description of elements, processes, and activities to represent the dynamics of a complete episode of mTBI care for use in modeling a TBI care system at the enterprise level. This includes an outline of the structure of a model or methodology to assist in planning for the alloca- tion of scarce TBI care providers in theater and in CONUS. Thus the approach structure must define care paths that specify which functions are necessary and when branching and feedback paths occur and their associated criteria, and where most time is consumed in the system. Pro- cess maps would be developed, as necessary, and a model representation would be chosen with defined parameters (Figure 6-1). Data requirements would be identified for estimating these param- eters, and sensitivity analyses would be carried out to verify and test the model. Validation of the model would be accomplished through evalu- ation of the model relative to baseline data. Resource-allocation experi- ments would then be performed to assess, for example, the effectiveness and resource requirements for alternative disease-management protocols or the effectiveness of alternative distributions of a limited number of providers among multiple echelons of care. Data requirements for this analysis plan are divided into four c Â ategories: 1.Â known aspects of mTBI with available data 2.Â known aspects of mTBI with no available data
From Level I Level II *Level III Evacuation Decision Red Flags: 1. Progressively declining level of Conduct evaluation: consciousness/neurological exam Perform entire MACE Evaluate for red flags 2. Pupillary asymmetry 3. Seizures 4. Repeated vomiting Yes Treatment: Are Level III 1. Headache management, use Acetominophen. Evacuation to Level III (as red flags* 2. Avoid tramadol, narcotics, NSAIDâs, ASA, or operational considerations allow) present? other platelet inhibitors until CT confirmed negative. 3. Give an educational sheet to all positive No mild TBI patients. Any Is MACE score deterioration, Yes for Items IX to XIII Yes Observe â up to 7 days persistent symptoms, Evacuation to Level III under 25, or are there (command decision) or positive findings on (as operational any symptoms from repeat MACE considerations allow) Item VIII? after 7 days? No No Positive Repeat testing in 24 hours Positive symptoms or evacuate to Level III symptoms with exertional Yes (command decision) with exertional Yes exercise testing exercise testing for 5 minutes for 5 minutes (sit -ups push-ups, , (sit-ups, push-ups, run)? run)? No No RTD RTD FIGURE 6-1 Sample mTBI care-process map for Level II treatment facilities. Source: DVBIC Working Group, 2006. 111 Figure 6-1.eps
112 Systems Engineering to Improve Traumatic Brain Injury CARE 3.Â aspects of mTBI with a recognized lack of understanding 4.Â aspects of mTBI yet to be identified Present knowledge and currently available data are as follows: 11.2 percent of individuals surveyed in the military have reported mTBI; 50 percent of these individuals accessed care; and 10 to 20 percent of reports of mTBI are post-deployment (Labutta, 2008). In addition, both in theater and post-deployment reporting are currently designed to generate false positives. Ninety percent of care providers in the field use MACE,20 and 85 percent of civilian non-blast patients recover from their injuries within three months. Currently, we do not have substantiating data on the complete path of care in MHS, the effectiveness of rehabilitation therapy, or outcomes for those who return to work. Apparent issues identified with no supporting knowledge include the extent to which a TBI report indicates the presence of co-morbidities and the progression of TBI symptoms without treatment. Finally, there are still aspects of mTBI about which there is no information and no treatment. A major assumption for this analysis plan is that the plans developed by Groups A through D, including the development of new TBI knowl- edge, the detection and screening of TBI conditions, the coordination and communication of TBI care, and the forecasting of the demand for care, will all accomplish their objectives. Nevertheless, work on imple- menting the Group E analysis plan can proceed in parallel with work on these other initiatives, as long as we allow for refinements in the model structure and data as additional information becomes available. Sensitivity analysis with early versions of the model can be used to help prioritize the need for additional information. In addition, standard modeling or computational engines involving standard representations must be used, and required data in existing information systems must be âcapturable.â Critical constraints of modeling for this analysis plan include the lack of knowledge about care paths and results, the lack of baseline comparison data, and problems with determining the sample size be- cause of the large number of care paths. The model would normally be calibrated against a baseline of performance of the system, but it is 20 MACE stands for Military Acute Concussion Evaluation. For more information, see http://www.dvbic.org/pdfs/DVBIC_instruction_brochure.pdf (accessed September 29, 2008).
Suggestions for Analysis Plans by Working Groups 113 not currently clear what the baseline performance is or how one would assess it. Because of the large variety of care paths, many samples of patients with particular combinations of symptoms are too small to be statistically meaningful. Other constraints on the implementation of this plan include (1)Â acceptance of the model in the military culture, by individuals, and by the public and (2)Â resource constraints because moderate and severe TBI consume the majority of resources for TBI care and management, leaving few resources for mTBI care. Performance, outcome, and utility metrics would include the input of a population from each of the 48 care program categories; an estimate of patient coverage (the percentage of individuals detected and treated appropriately and the percentage missed); and patient safety outcomes in terms of work-related activities, treatment accessibility (how far patients must travel for treatment), percentage returning to work, and quality of life. Other metrics are the costs and trade-offs of resource requirements (e.g., people, facilities, funding), as well as the people and time required to operate and maintain the model (including collecting data, estimat- ing parameters, and conducting experiments). Overall, the usefulness of this analysis plan is that it could create interactive models capable of generating answers to various questions in only a few months. A necessary task for the execution of this analysis plan is a value- stream analysis for chronic mTBI. Currently, hundreds of soldiers per month âscreenâ positive for self-reported blast injuries and symptoms, with roughly 50 percent of these diagnosed as symptoms related to mTBI. A value-stream analysis would involve the identification of a cohort of patients with symptoms three months after injury and the tracking of these individuals through the system for three months post- diagnosis by walking with them through the process and understanding what they experience. Then a larger set of patients would be tracked through the MHS information system and surveyed in comparison to all other patients. This would enable researchers to observe what actually happens to patients at each step and to determine the value added of each step in terms of insight and treatment for particular outcomes. The path of each patient would be mapped through the system, with durations and branching frequencies. Completion of the model and essential outputs for users would take about six months. Following the preliminary plan described above, a series of model- ing spirals would have to be defined that would âspiralâ through the
114 Systems Engineering to Improve Traumatic Brain Injury CARE development of process maps for âas isâ and âto beâ care of patients. A model representation, which would include a disease model showing the progression of mTBI, an organizational model showing how relevant or- ganizations function, and a care management model showing how care is managed by these organizations, would then be selected. Parameters for the model representation would have to be defined, and data require- ments would have to be identified for estimating these parameters. Once the model has been verified and validated through preliminary testing and evaluations, resource-allocation experiments could be performed. The proposed model could be used to inform policy development, respond to congressional inquiries, and identify resource implications for policy changes in terms of people, training, education, and money. In addition, the model could further an understanding of the implications of health quality outcomes, determine optimum allocations of limited resources, and ascertain what is unknown or cannot yet be imagined in relation to emergent behaviors following mTBI. Users of the model would include DOD, Defense Centers of Excellence, MHS, and VA. Outputs of the model would be useful to policy makers, the Govern- ment Accountability Office and Inspector Generals, and authorizers and appropriators. The expected outputs for the proposed model include (1)Â the iden- tification of processes that need improvement and suggestions for im- proving them; (2)Â the identification of, and priorities for, processes that should be created; and (3)Â the identification of critical data that should be collected. Priorities for data collection include information identified through sensitivity analyses, processes that truly impact outcomes, and areas in which exact numbers are needed. Implementation actions include acceptance of the model by DOD and the definition of a course of action, the development of a business case for return on investment of such an initiative, and the elaboration of how current modeling investments could yield much larger future benefits. These benefits would include both longer term returns in reduced workloads and lower costs for MHS and VA health care provid- ers, as well as the shorter term benefits of informed research road maps; analytically defensible investment strategies; credible, compelling risk- management strategies; and prioritized data gathering for high-leverage information. The requirements for this initiative are estimated to be six months for the value-stream analysis, including data collection and direct
Suggestions for Analysis Plans by Working Groups 115 o Â bservation, and an additional six months for each subsequent spiral. The nature of the spirals will depend on the questions that emerge from previous spirals, assuming a collaborative effort and broader coverage of phenomena. Core competencies include proficiency in working with MHS, VA, mTBI, and modeling systems; the value-stream analysis will require contributions from social scientists. The total resources required would equal the value of the person years per month times six months times N + 1, where N represents the number of spirals. SUMMARY The challenges of the TBI workshop from the perspective of OSE are captured in Figure 6â2. Before OSE methods can be brought to bear on TBI care, there must be at least a preliminary understand- ing of the relationships between blast and concussive events and TBIs subject to the conditions of delivery or occurrence and the state of the soldier who is injured. The development of this understanding was addressed by Working Group A in an approach focused on the use of diagnostic and screening tools to establish pre- and post-event baselines, as well as conducting basic research on blast and concus- sive effects. The current MHS TBI care delivery system must be better speci- fied and understood for OSE tools and methods to be used effectively. The complex military health care delivery system includes facilities, medical logistical support, and personnel of the MHS, VA, and civilian health care systems, as well as the families of soldiers suffering from TBI and the soldiers themselves. One of the basic challenges associ- ated with the delivery of care in this system is patient tracking and case management. Working Group C suggested an approach to the development of an information system for tracking, monitoring, and cueing care delivery for all TBI patients. The approach focuses on combining the integra- tion and augmentation of existing databases and a communication system that would ensure access to information and the dissemination of information to appropriate parties; the system architecture would be compatible with the care delivery system. Finally, Group A noted that data available outside the military TBI domain, in the form of records of concussive and other closed-head
Real World Model World 116 Improved understanding of effects Identify blast effects on Develop model Assess value brain function (A) for diagnosis of prevention (B) (D) Diagnostic Improved Changes in methods screening prevalence Structure TBI information Improved Evaluate screening Estimate current and system and methodology Prevalence for coordinated care diagnostic processes future prevalence (C) processes (B) (D) Improved screening processes Patient stream Burden of disease Disease history of patient stream Information on Model episode of care Develop database on natural history Develop disease model and allocation of civilian events Impacts on scarce resources (A) (A) disease history (E) Information on effects of treatment Improved treatment processes and associated resource requirements FIGURE 6-2 Interrelationships among suggestions for analysis plans developed by participants in Working Groups A through E. Figure S-1.eps landscape
Suggestions for Analysis Plans by Working Groups 117 injuries and cases of TBI in the civilian world, could contribute to the development of a more detailed understanding of the course of TBI and the effectiveness of alternative treatments. The group suggested an ap- proach for integrating civilian and military data for use in the military environment. The remaining blocks in Figure 6-2 show typical tasks, largely inter- related, that can be addressed by OSE techniques and methodologies. The development of a quantitative disease model is key to the evaluation of long-term demand for care, strategies for treatment, and the design (and costs) of a high-quality, efficient care delivery system. Working Group A developed an approach to modeling the course of TBI as a finite state-space stochastic process in which patients transition from state to state as a function of additional (non-TBI) trauma, treatment, and the long-term impact of trauma, with co-morbidities impacting the definition and occupancy times of any given state. Group A also developed a plan for using a survey methodology integrated with data mining to support the definition of states, the estimations of transition probabilities, and the distributions of occupancy times. Parameters in the quantitative disease model include diagnosis and screening characteristics, state definitions, and state transitions. Work- ing Group B outlined the development of a series of models that would quantitatively describe and evaluate current practices and then optimize the screening process. The analysis of data through data mining and the execution of surveys would be essential to the development of these models. Markov decision theory, Bayesian networks, influence diagrams, and simulation were cited as viable candidates for evaluating and design- ing TBI diagnostic and screening processes (see Appendix H). To assess the capacity of MHS to treat TBI, one must first under- stand and be able to predict the âdemandâ on the system. Working Group D described the difficulties of estimating demand, which is complicated because, in many instances, the presence of TBI is not rec- ognized or is not reported promptly in theater and can be missed in post- deployment interviews. The group observed that historical data could be analyzed to develop statistical estimates of the number of mTBIs in the current population of military personnel who have served or are serving in Iraq and Afghanistan. From there, the group outlined a methodology for forecasting future mTBI cases, taking into account conditions in the military theater, threats, and role-based exposures. The group also addressed the challenge of assessing the value of TBI prevention efforts.
118 Systems Engineering to Improve Traumatic Brain Injury CARE The value of specific investments in prevention could be compared us- ing methods of estimating the reduction of TBI incidence as well as the cost savings to the health care system and the reduced burden on soldiers and their families. Given a quantitative disease model and an understanding of the effectiveness, availability, and costs of treatment, it is possible to design and evaluate care delivery from the perspective of individual patients, a patient population, and the entire health care enterprise. Working Group E designed an approach to the development of an enterprise-level health care delivery model with the goal of addressing, quantitatively, a broad spectrum of TBI treatment capacity, organizational and resource allocation issues to support decisions on policy, and design of the health care enterprise. The suggestions for analysis plans developed by the five working groups indicate how OSE methods and tools could contribute to meet- ing the challenges of delivering effective, efficient, high-quality TBI care and management. Particular quantitative methods and models could provide insights into the design of diagnostic and screening processes, the delivery of care, the sizing of facilities, and the design of the overall health care delivery complex to meet current and future demands. As depicted in Figure 6-2, the challenges addressed by the OSE analysis plans are interrelated with the outputs of each analysis plan potentially providing important inputs to the development of one or more of the other plans. Collectively these illustrative plans address many of the TBI stakeholder issues identified during the planning phase of the workshop (Appendix B). The suggestions and assumptions of the working groups identified underlying cross-cutting dependencies important to the development and application of OSE methods and tools to TBI care: â¢ an assumption that sufficient reliable data are available for the development of useful initial versions of all of the plans suggested by the working groups and that additional reliable data could be generated to improve and refine these approaches and provide more comprehensive and precise support to meet MHS needs â¢ a recognition of the need for standardized, detailed coding to record TBI symptoms, injuries, and (possibly) treatments more accurately
Suggestions for Analysis Plans by Working Groups 119 â¢ the importance of data sharing and interoperability of data bases relevant to TBI diagnosis, treatment, measurement, and prediction â¢ the need for more extensive, detailed maps of care paths and pro- cesses and associated information, patient, provider, and material flows for TBI care within the MHS The ideas and concepts introduced during the workshop may be helpful to DOD leaders working to refine and improve DODâs system of health care delivery, both at the individual patient-provider level and at the enterprise level. Applications of OSE concepts, tools, and methods have the potential to contribute to improvements in care, not only for TBI patients but for all patients cared for by MHS. References Cope, D.N., N.H. Mayer, and L. Cervelli. 2005. Development of systems of care for persons with traumatic brain injury. Journal of Health Trauma Rehabilitation. Focus on Clinical Research and Practice 20(2): 128â142. DVBIC Working Group (Defense and Veterans Brain Injury Center Working Group on the Acute Management of Mild Traumatic Brain Injury in Military Operational Settings). 2006. Clinical Practice Guideline and Recommendations 22 December 06. Available online at http://dvbic.org/public_html/pdfs/clinical_practice_guideline_recommendations. pdf (accessed August 7, 2008). Guler, I., A. Tunca, and E. Gulbandilar. 2008. Detection of traumatic brain injuries using fuzzy logic algorithm. Expert Systems Applications 34(2): 1312â1317. Labutta, R.J. 2008. Medical Aspects of Traumatic Brain Injury (TBI). Presentation at the Workshop on Harnessing Operational Systems Engineering to Improve Traumatic Brain Injury Care in the Military Health System, National Academies, Washington, D.C. June 11, 2008. Sayer, N. 2006. Department of Veterans Affairs Quality Enhancement Research Initiative. 2006 Strategic Plan. Available online at http://www.hsrd.minneapolis.med.va.gov/pdf/ PT_Strategic Plan.pdf (accessed September 29, 2008).