Cover Image

PAPERBACK
$45.00



View/Hide Left Panel

The Evolution of Brain-Computer Interfaces

ERIC C. LEUTHARDT
Washington University



ABSTRACT

The notion that a computer can decode brain signals to infer the intentions of a human subject and then enact those intentions directly through a machine is emerging as a realistic technical possibility. These types of devices are known as brain-computer interfaces or BCIs. The evolution of these neuroprosthetic technologies could have significant implications for patients with motor disabilities by enhancing their ability to interact and communicate with their environment. Classically, the cortical physiology that has been most investigated and utilized for device control has been brain signals from the primary motor cortex. To date, this classic motor physiology has been an effective substrate for demonstrating the potential efficacy of BCI-based control. Emerging research in cortical physiology, however, now stands to further enhance our understanding of the cortical physiology underpinning human intent and provides further signals for more complex brain-derived control. In this review, we discuss the current status of BCIs and detail the emerging research trends that stand to further augment clinical application in the future.

INTRODUCTION

The notion that the brain can be directly accessed to allow a human being to control an external device with thoughts alone is emerging as a real option for patients with motor disabilities. This area of study, known as neuroprosthetics, has sought to create devices, known as brain-computer interfaces (BCIs), that acquire brain signals and translate them to machine commands such that they reflect



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 123
The Evolution of Brain-Computer Interfaces eric c. leuthArdt Washington University ABSTRACT The notion that a computer can decode brain signals to infer the intentions of a human subject and then enact those intentions directly through a machine is emerging as a realistic technical possibility. These types of devices are known as brain-computer interfaces or BCIs. The evolution of these neuroprosthetic tech - nologies could have significant implications for patients with motor disabilities by enhancing their ability to interact and communicate with their environment. Classically, the cortical physiology that has been most investigated and utilized for device control has been brain signals from the primary motor cortex. To date, this classic motor physiology has been an effective substrate for demonstrat- ing the potential efficacy of BCI-based control. Emerging research in cortical physiology, however, now stands to further enhance our understanding of the cortical physiology underpinning human intent and provides further signals for more complex brain-derived control. In this review, we discuss the current status of BCIs and detail the emerging research trends that stand to further augment clinical application in the future. INTRODUCTION The notion that the brain can be directly accessed to allow a human being to control an external device with thoughts alone is emerging as a real option for patients with motor disabilities. This area of study, known as neuroprosthetics, has sought to create devices, known as brain-computer interfaces (BCIs), that acquire brain signals and translate them to machine commands such that they reflect 123

OCR for page 123
124 FRONTIERS OF ENGINEERING the intentions of the user. In the past 20 years, the field has progressed rapidly from fundamental neuroscientific discovery to initial translational applications. Examples are seen in the seminal discoveries by Georgopoulus and Schwartz that neurons in the motor cortex, when taken as a population, can predict the direction and speed of arm movements in monkeys (Georgopoulos et al., 1982, 1986; Moran and Schwartz, 1999a). In the subsequent decades, these findings were translated to increasing levels of brain-derived control in monkeys and to preliminary human clinical trials (Hochberg et al., 2006; Taylor et al., 2002). Fundamental to the evo - lution of neuroprosthetic application, this brain-derived control is dependent on the emerging understanding of cortical physiology as it encodes information about intentions. In recent years, an emerging understanding of how the cortex encodes motor and nonmotor intentions, sensory perception, and the role that cortical plasticity plays in device control have led to new insights in brain function and BCI application. These new discoveries stand to further expand the potential of neuroprosthetics in regards to both control capability and patient populations that will be served. In this review, we provide an overview of current BCI modalities and of emerging research on the use of nonmotor areas for BCI applications, and we assess their potential for clinical impact. BRAIN-COMPUTER INTERFACE: DEFINITION AND ESSENTIAL FEATURES A BCI is a device that can decode human intent from brain activity alone in order to create an alternate communication channel for people with severe motor impairments. More explicitly, a BCI does not require the “brain’s normal output pathways of peripheral nerves and muscles” to facilitate interaction with one’s environment (Wolpaw et al., 2000, 2002). A real-world example of this would entail a quadriplegic subject controlling a cursor on a screen with signals derived from individual neurons recorded in the primary motor cortex without the need of overt motor activity. It is important to emphasize this point. A true BCI creates a completely new output pathway for the brain. As a new output pathway, the user must have feedback to improve the performance of how one alters one’s electrophysiological signals. Similar to the development of a new motor skill (e.g., learning to play tennis), there must be continuous alteration of the subject’s neuronal output. The output should be matched against feedback from intended actions such that the subject’s output (swinging the tennis racket or altering a brain signal) can be tuned to optimize performance toward the intended goal (getting the ball over the net or moving a cursor toward a target). Thus, the brain must change its signals to improve performance, but additionally the BCI may also be able to adapt to the changing milieu of the user’s brain to further optimize functioning. This dual adaptation requires a certain level of training and a learning curve, both for the user and for

OCR for page 123
125 THE EVOLUTION OF BRAIN-COMPUTER INTERFACES the computer. The better the computer and subject are able to adapt, the shorter the training that is required for control. There are four elements essential to the practical functioning of a BCI plat- form (Figure 1): 1. Signal acquisition, the BCI system’s recorded brain signal or information input; 2. Signal processing, the conversion of raw information into a useful device command; 3. Device output, the overt command or control functions that are admin- istered by the BCI system; and 4. Operating protocol, the manner in which the system is altered and turned on and off (Wolpaw et al., 2002). All of these elements play in concert to manifest the user’s intention in his or her environment. Signal acquisition is some real-time measurement of the electrophysiological state of the brain. This measurement of brain activity is usually recorded via electrodes, but this is by no means a theoretical requirement. These electrodes can be either invasive or noninvasive. The most common types of signals include electroencephalography (EEG), electrical brain activity recorded from the scalp FIGURE 1 Essential features and components of a BCI. There are four essential elements to the practical functioning of a brain computer interface platform: (1) signal acquisition, the BCI system’s recorded brain signal or information input; (2) signal processing, the conversion of raw information into a useful device command; (3) device output, the overt command or control functions that are administered by the BCI system; and (4) operating protocol, the manner in which the system is turned on and off, and the way the user or a technical assistant adjusts parameters of the previous three steps in converting intentions to machine commands. All of these elements play in concert to manifest the user’s intention in his or her environment (Schalk et al., 2004b). Source: Leuthardt et al. (2009).

OCR for page 123
126 FRONTIERS OF ENGINEERING (Elbert et al., 1980; Farwell and Donchin, 1988; Freeman et al., 2003; Pfurtscheller et al., 1993; Sutter, 1992; Vidal, 1977); electrocorticography (ECoG; Leuthardt et al., 2004, 2005), electrical brain activity recorded beneath the skull (Leuthardt et al., 2004, 2005; Schalk et al., 2004a); field potentials, electrodes monitoring brain activity from within the parenchyma (Andersen et al., 2004); and “sin - gle units,” microelectrodes monitoring individual neuron action potential firing (Georgopoulos et al., 1986; Kennedy and Bakay, 1998; Laubach et al., 2000; Taylor et al., 2002). Figure 2 shows the relationship of the various signal platforms in terms of anatomy and population sampled. Once acquired, the signals are then digitized and sent to the BCI system for further interrogation. In the signal-processing portion of BCI operation, there are two essential functions: feature extraction and signal translation. The first function extracts significant identifiable information from the gross signal, and the second converts that identifiable information into device commands. The process of converting a raw signal into one that is meaningful requires a complex array of analyses. These techniques can vary from assessment of frequency power spectra, event-related potentials, and cross-correlation coefficients for analysis of EEG or ECoG signals to directional cosine tuning of individual neuron action potentials (Levine et al., 2000; Moran and Schwartz, 1999a; Pfurtscheller et al., 2003). The impetus for these methods is to determine the relationship between an electrophysiologic event and a given cognitive or motor task. As an example, after recordings are made from an ECoG signal, the BCI system must recognize that a signal altera - tion has occurred in the electrical rhythm (feature extraction) and then associate that change with a specific cursor movement (translation). As mentioned above, it is important that the signal processing be dynamic such that it can adjust to the FIGURE 2 Signals for BCI. Three general categories of signals that are used for BCI application and their anatomic location relative to the brain and its respective covering layers. EEG, electroencephalography; ECoG, electrocorticography. Source: Leuthardt et al. (2009).

OCR for page 123
127 THE EVOLUTION OF BRAIN-COMPUTER INTERFACES changing internal signal environment of the user. In regards to the actual device output, this is the overt action that the BCI accomplishes. As in the previous exam- ple, this can result in moving a cursor on a screen; other possibilities are choosing letters for communication, controlling a robotic arm, driving a wheelchair, or controlling some other intrinsic physiologic process such as moving one’s own limb or controlling one’s bowel and bladder sphincters (Leuthardt et al., 2006a). An important consideration for practical application is the overall operating protocol. This refers to the manner in which the user controls how the system functions. The “how” includes such things as turning the system on or off, controlling the kind of feedback and how fast it is provided, how quickly the system implements commands, and switching between various device outputs. These elements are critical for BCI functioning in the real-world application of these devices. In most current research protocols, these parameters are set by the investigator. In other words, the researcher turns the system on and off, adjusts the speed of interaction, or defines very limited goals and tasks. These are all things that the user will need to be able to do by him- or herself in an unstructured applied environment. CURRENT BCI PLATFORMS There are currently three general categories of BCI platforms that have been put forward as possible candidates for clinical application. These categories are primarily determined by the source from which the controlling brain signal is derived. The first category uses EEG, which involves brain signals acquired from the scalp. The second category, referred to as “single-unit systems,” uses intraparenchymal microelectrodes that detect action potential firings of individual neurons. The third is an intermediate modality in which electrodes acquire signal from the cortical surface directly (either above or below the dura). The current status of each of these platforms is briefly reviewed in terms of level of control, surgical considerations, and current clinical populations served. EEG-Based Systems EEG-based BCIs use electrical activity recorded from the scalp (Birbaumer et al., 1999; Blankertz et al., 2006; Farwell and Donchin, 1988; Kübler et al., 2005; McFarland et al., 1993, 2008a; Millan et al., 2004; Muller et al., 2008; Pfurtscheller et al., 1993, 2000; Sutter, 1992; Vaughan et al., 2006; Wolpaw and McFarland, 1994, 2004; Wolpaw et al., 1991). Most BCI studies in humans used EEG, probably because this recording method is convenient, safe, and inexpensive. EEG has a relatively poor spatial resolution. This is because a large brain area must be involved to generate the necessary detectable signals (Freeman et al., 2003; Srinivasan et al., 1998). Despite this limitation, signals relevant to BCI research can still be found in the EEG. This includes modulations of mu

OCR for page 123
128 FRONTIERS OF ENGINEERING (8-12 Hz) or beta (18-25 Hz) rhythms produced by sensorimotor cortex. These rhythms show nonspecific changes (typically decreases in amplitude) related to movements and movement imagery. They do not contain specific informa - tion about the details of movements, such as the position or velocity of hand movements. This may be an important limitation because signals associated with specific movement parameters are typically used in BCI systems based on action potential firing rates. Another issue of EEG recordings is that the detected amplitudes are very small. This makes them susceptible to artifacts created by sources outside the brain such as electromyographic signals produced by muscle contractions. Despite these potentially limiting issues, EEG-based BCIs have been shown to support higher performance than is often assumed, including accurate two-dimensional (McFarland et al., 2008a; Wolpaw and McFarland, 2004) and even three-dimensional control of a computer cursor (McFarland et al., 2008b). To date, the large majority of clinical application of BCI technologies for people with severe motor disabilities have been demonstrated using EEG (Kübler et al., 2005; Nijboer et al., 2008; Vaughan et al., 2006). Ultimately, this intrinsic lack of signal robustness may have important implications for chronic application of BCI systems in real-world environments. BCI systems based on EEG typically require substantial training (Birbaumer, 2006; Wolpaw and McFarland, 2004) to achieve accurate one- or two-dimensional device control (about 20 or 50 thirty-minute training sessions, respectively), although some reports have reported training requirements that are shorter (Blankertz et al., 2006). These shortcomings of noise sensitivity and prolonged training are fundamental limitations in the scalability of widespread clinical application of EEG-based BCIs. In summary, EEG has been shown to support much higher performance than previously assumed and is currently the only modality that has been shown to actually help people with paralysis. However, because of its important limita- tions, it is currently not clear to what extent EEG-based BCI performance, in the laboratory and in clinical settings, can be further enhanced. Single-Neuron-Based Systems From a purely engineering point of view, the optimal method to extract elec - trical information from the brain would be to place a series of small recording electrodes directly into the cortical layers (1.5–3 mm) to record signals from indi - vidual neurons. This, in essence, is what single-unit action potential BCI systems do, and they have been very successful for limited time periods in both monkeys (Carmena et al., 2003; Serruya et al., 2002; Taylor et al., 2002; Velliste et al., 2008) and humans (Hochberg et al., 2006; Kennedy and Bakay, 1998). To extract single- unit activity, small microelectrodes having ~20-micron-diameter tips are inserted in the brain parenchyma where relatively large (e.g., 300 microvolt) extracellular action potentials are recorded from individual neurons from 10-100 microns away. These signals are usually bandpass filtered from 300 to 10,000 Hz and then passed

OCR for page 123
129 THE EVOLUTION OF BRAIN-COMPUTER INTERFACES through a spike discriminator to measure spike time occurrences. The firing rates of individual neurons are computed in 10- to 20-millisecond bins and “decoded” to provide a high-fidelity prediction to control either a computer cursor or robot endpoint kinematics (Georgopoulos et al., 1986; Moran and Schwartz, 1999b; Wang et al., 2007). Given its high spatial resolution (100 microns) as well as its high temporal resolution (50-100 Hz), this modality arguably provides the highest level of control in BCI applications. Unfortunately, there are two major problems with single-unit BCIs. First, the electrodes must penetrate into the parenchyma, where they cause local neural and vascular damage (Bjornsson et al., 2006). Second, single-unit action poten - tial microelectrodes are very sensitive to encapsulation. Insertion of penetrating devices in the brain parenchyma damages neurons and vasculature, which can initiate a cascade of reactive cell responses, typically characterized by activation and migration of microglia and astrocytes toward the implant site (Bjornsson et al., 2006). The continued presence of devices promotes formation of a sheath composed partly of these reactive astrocytes and microglia (Polikov et al., 2005; Szarowski et al., 2003). This reactive sheath can have numerous deleterious effects, including neural cell death and an increased tissue resistance that electri - cally isolates the device from the surrounding neural tissue (Biran et al., 2005; Szarowski et al., 2003; Williams et al., 2007). Research into novel biomaterial coatings and/or local drug delivery systems that may reduce the foreign body response to implanted electrodes is ongoing but, to date, is far from clinical application (Abidian and Martin, 2008; Seymour and Kipke, 2007; Spataro et al., 2005). Until these issues are solved, this remains a limitation for developing a long-term BCI system based on single-unit activity. Electrocorticography-Based Systems Over the past 5 years, the use of ECoG as a signal platform for BCI has gained mounting enthusiasm as a more practical and robust platform for clinical application. As detailed above, both EEG- and single-unit-based systems have been impeded for large-scale clinical application. This is either due to prolonged user training and poor signal-to-noise limitations with EEG, or due to inability to maintain a consistent signal with current single-unit constructs (Bjornsson et al., 2006; Szarowski et al., 2003; Wolpaw and McFarland, 2004). The use of ECoG has been posited to be an ideal trade-off for practical implementation (Leuthardt et al., 2004). When compared to EEG, the signal is substantially more robust. Its magnitude is typically five times larger, its spatial resolution is much greater (0.125 versus 3.0 cm for EEG), and its frequency bandwidth is significantly higher (0-500 Hz versus 0-40 Hz for EEG) (Boulton et al., 1990; Freeman et al., 2003; Srinivasan et al., 1998). Of particular note, the access to higher-frequency band - widths carries particularly useful information amenable to BCI operation (Gaona et al., 2011). Many studies have demonstrated that different frequency bands

OCR for page 123
130 FRONTIERS OF ENGINEERING carry specific and anatomically distinct information about cortical processing. The lower-frequency bands known as mu (8-12 Hz) and beta (18-25 Hz), which are detectable with EEG, are thought to be produced by thalamocortical circuits and show broad anatomic decreases in amplitude in association with actual or imag- ined movements (Huggins et al., 1999; Levine et al., 1999; Pfurtscheller et al., 2003; Rohde et al., 2002). The higher frequencies only appreciable with ECoG, also known as gamma band activity, are thought to be produced by smaller cortical assemblies. Gamma activity shows close correlation with action potential firing of tuned cortical neurons in the primary motor cortex in monkey models (Heldman et al., 2006). Additionally, these high-frequency changes have been associated with numerous aspects of speech and motor function in humans (Chao et al., 2010; Crone et al., 1998, 2001a, 2001b; Gaona et al., 2011; Leuthardt et al., 2004; Schalk et al., 2007). Beyond higher information content, since the ECoG signal is recorded from larger electrodes that do not penetrate the brain, these constructs should have a higher likelihood for long-term clinical durability. This expectation of good long-term stability of ECoG sensors is supported by some pathologic and clinical evidence. For example, in cat, dog, and monkey models, long-term sub - dural implants showed minimal cortical or leptomeningeal tissue reaction while maintaining prolonged electrophysiologic recording (Bullara et al., 1979; Chao et al., 2010; Loeb et al., 1977; Margalit et al., 2003; Yuen et al., 1987). Addition- ally, preliminary work in humans using the implantable NeuroPace device for the purpose of long-term subdural electrode monitoring for seizure identification and abortion has also been shown to be stable (Vossler et al., 2004). The use of ECoG for BCI applications has been primarily studied in motor- intact patients with intractable epilepsy requiring invasive monitoring. Similar to EEG-based BCI systems, the ECoG approach has primarily focused on the use of changes in sensorimotor rhythms from motor cortex. What has been distinct, how- ever, has been the access to the higher-frequency gamma rhythms with ECoG. Use of these higher-frequency rhythms has provided a significant advantage in regards to training requirements and multidimensional control. In 2004, Leuthardt et al. demonstrated the first use of ECoG in closed-loop control in a one-dimensional cursor control task with minimal training requirements (under 30 minutes). In additional experiments, the same group and others have demonstrated that specific frequency alterations encode very specific information about hand and arm move - ments (Leuthardt et al., 2004; Pistohl et al., 2008; Sanchez et al., 2008; Schalk et al., 2007). In 2006, Leuthardt et al. further demonstrated that ECoG control using signal from the epidural space was also possible (Leuthardt et al., 2006b). Schalk et al. (2008) showed that ECoG signals can be used for two-dimensional control whose performance was within the range of that shown before using invasive single-unit systems. Because the electrode arrays cover broad regions of cortex, several groups have begun to explore alternate cognitive modalities and their cortical physiologies to expand BCI device control. Felton et al. (2007)

OCR for page 123
131 THE EVOLUTION OF BRAIN-COMPUTER INTERFACES showed that, in addition to motor imagery, sensory imagery could also be used for device control. The same group also demonstrated that the auditory cortex could be trained to acquire simple control of a cursor (Wilson et al., 2006). Ramsey et al. (2006) showed that higher cognitive functions, such as working memory in the dorsal lateral prefrontal cortex, can also be used for effective device operation. Recently, Leuthardt et al. (2011) demonstrated that phonemic content taken from speech networks could also be used for simple device control. Taken together, these studies show that ECoG signals carry a high level of specific cortical information, and that these signals can allow a user to gain control rapidly and effectively. It is worth noting that these control paradigms have not been extended to motor-impaired subjects thus far. How these cortical signals will be affected in the setting of a spinal cord injury or ALS has not been explicitly tested. CONCLUSIONS The field of neuroprosthetics is growing rapidly. The cortical physiology that underpins the manner in which a human brain encodes intentions is beginning to be understood. This will have a significant impact in augmenting function for those with various forms of motor disabilities. As research stretches beyond motor physiology, the field of neuroprosthetics now stands to further expand in capabil - ity and in diversity of clinical populations served. The evolving understanding of cortical physiology as it relates to motor movements, language function, and plasticity could all provide higher levels of complexity in brain-derived control. Given the rapid progression of these technologies over the past decade and the concomitant swift ascent of computer processing speeds, signal analysis tech - niques, and emerging ideas for novel biomaterials, neuroprosthetic implants will hopefully in the near future be as common as deep brain stimulators are today. The clinical advent of this technology will usher in a new era of restorative neuro- surgery and new human-machine interfaces. ACKNOWLEDGEMENTS The author would like to acknowledge the generous support from the following: the James S. McDonnell Foundation, the Department of Defense (W911NF-07-1-0415 & W911NF-08-1-0216 – Leuthardt), the National Insti- tutes of Health (NINDS NIH R01-EB000856-06), and the Children’s Discovery Institute. Financial Disclosures Dr. Leuthardt owns stock in the company Neurolutions.

OCR for page 123
132 FRONTIERS OF ENGINEERING REFERENCES Abidian, M.R., and D.C. Martin. 2008. Experimental and theoretical characterization of implant - able neural microelectrodes modified with conducting polymer nanotubes. Biomaterials 29(9):1273–1283. Andersen, R.A., J.W. Burdick, S. Musallam, B. Pesaran, and J.G. Cham. 2004. Cognitive neural prosthetics. Trends in Cognitive Sciences 8(11):486–493. Biran, R., D.C. Martin, and P.A. Tresco. 2005. Neuronal cell loss accompanies the brain tissue response to chronically implanted silicon microelectrode arrays. Experimental Neurology 195(1):115–126. Birbaumer, N., N. Ghanayim, T. Hinterberger, I. Iversen, B. Kotchoubey, A. Kübler, J. Perelmouter, E. Taub, and H. Flor. 1999. A spelling device for the paralysed. Nature 398(6725):297–298. Birbaumer, N. 2006. Brain-computer-interface research: Coming of age. Clinical Neurophysiology 117(3):479–483. Bjornsson, C.S., S.J. Oh, Y.A. Al-Kofahi, Y.J. Lim, K.L. Smith, J.N. Turner, S. De, B. Roysam, W. Shain, and S.J. Kim. 2006. Effects of insertion conditions on tissue strain and vascular damage during neuroprosthetic device insertion. Journal of Neural Engineering 3(3):196–207. Blankertz, B., G. Dornhege, M. Krauledat, K.R. Muller, V. Kunzmann, F. Losch, and G. Curio. 2006. The Berlin Brain-Computer Interface: EEG-based communication without subject training. IEEE Transactions on Neural Systems and Rehabilitation Engineering 14(2):147–152. Boulton, A.A., G.B. Baker, and C.H. Vanderwolf. 1990. Neurophysiological Techniques, II: Applica- tions to Neural Systems. Clifton, N.J.: Humana Press. Bullara, L.A., W.F. Agnew, T.G. Yuen, S. Jacques, and R.H. Pudenz. 1979. Evaluation of electrode array material for neural prostheses. Neurosurgery 5(6):681–686. Carmena, J., M. Lebedev, R. Crist, J. O’Doherty, D. Santucci, D. Dimitrov, S. Patil, C. Henriquez, and M. Nicolelis. 2003. Learning to control a brain-machine interface for reaching and grasping by primates. PLoS Biology 1(2):193–208. Chao, Z.C., Y. Nagasaka, and N. Fujii. 2010. Long-term asynchronous decoding of arm motion using electrocorticographic signals in monkeys. Frontiers in Neuroengineering 3:3. Crone, N.E., D.L. Miglioretti, B. Gordon, and R.P. Lesser. 1998. Functional mapping of human sen - sorimotor cortex with electrocorticographic spectral analysis. II. Event-related synchronization in the gamma band. Brain 121(Pt 12):2301–2315. Crone, N.E., D. Boatman, B. Gordon, and L. Hao. 2001a. Induced electrocorticographic gamma activ - ity during auditory perception. Clinical Neurophysiology 112(4):565–582. Crone, N.E., L. Hao, J. Hart, Jr., D. Boatman, R.P. Lesser, R. Irizarry, and B. Gordon. 2001b. Electro - corticographic gamma activity during word production in spoken and sign language. Neurology 57(11):2045–2053. Elbert, T., B. Rockstroh, W. Lutzenberger, and N. Birbaumer. 1980. Biofeedback of slow cortical potentials. I. Electroencephalography and Clinical Neurophysiology 48(3):293–301. Farwell, L.A., and E. Donchin. 1988. Talking off the top of your head: Toward a mental prosthesis utilizing event-related brain potentials. Electroencephalography and Clinical Neurophysiology 70(6):510–523. Felton, E.A., J.A. Wilson, J.C. Williams, and P.C. Garell. 2007. Electrocorticographically controlled brain-computer interfaces using motor and sensory imagery in patients with temporary subdural electrode implants. Report of four cases. Journal of Neurosurgery 106(3):495–500. Freeman, W.J., M.D. Holmes, B.C. Burke, and S. Vanhatalo. 2003. Spatial spectra of scalp EEG and EMG from awake humans. Clinical Neurophysiology 114(6):1053–1068. Gaona, C., M. Sharma, Z. Freudenburg, J. Breshears, D. Bundy, J. Roland, D. Barbour, G. Schalk, and E. Leuthardt. 2011. Nonuniform high-gamma (60-500 Hz) power changes dissociate cognitive task and anatomy in human cortex. Journal of Neuroscience 31(6):2091–2100.

OCR for page 123
133 THE EVOLUTION OF BRAIN-COMPUTER INTERFACES Georgopoulos, A.P., J.F. Kalaska, R. Caminity, and J.T. Massey. 1982. On the relations between the direction of two-dimensional arm movements and cell discharge in primate motor cortex. Journal of Neuroscience 2(11):1527–1537. Georgopoulos, A.P., A.B. Schwartz, and R.E. Kettner. 1986. Neuronal population coding of movement direction. Science 233(4771):1416–1419. Heldman, D.A., W. Wang, S.S. Chan, and D.W. Moran. 2006. Local field potential spectral tuning in motor cortex during reaching. IEEE Transactions on Neural Systems and Rehabilitation Engineering 14(2):180–183. Hochberg, L.R., M.D. Serruya, G.M. Friehs, J.A. Mukand, M. Saleh, A.H. Caplan, A. Branner, D. Chen, R.D. Penn, and J.P. Donoghue. 2006. Neuronal ensemble control of prosthetic devices by a human with tetraplegia. Nature 442(7099):164–171. Huggins, J.E., S.P. Levine, S.L. BeMent, R.K. Kushwaha, L.A. Schuh, E.A. Passaro, M.M. Rohde, D.A. Ross, K.V. Elisevich, and B.J. Smith. 1999. Detection of event-related potentials for development of a direct brain interface. Journal of Clinical Neurophysiology 16(5):448–455. Kennedy, P.R., and R.A. Bakay. 1998. Restoration of neural output from a paralyzed patient by a direct brain connection. Neuroreport 9(8):1707–1711. Kübler, A., F. Nijboer, J. Mellinger, T.M. Vaughan, H. Pawelzik, G. Schalk, D.J. McFarland, N. Birbaumer, and J.R. Wolpaw. 2005. Patients with ALS can use sensorimotor rhythms to operate a brain-computer interface. Neurology 64(10):1775–1777. Laubach, M., J. Wessberg, and M.A. Nicolelis. 2000. Cortical ensemble activity increasingly predicts behaviour outcomes during learning of a motor task. Nature 405(6786):567–571. Leuthardt, E.C., G. Schalk, J.R. Wolpaw, J.G. Ojemann, and D.W. Moran. 2004. A brain- computer in- terface using electrocorticographic signals in humans. Journal of Neural Engineering 1(2):63–71. Leuthardt, E.C., K.J. Miller, G. Schalk, R.N. Rao, and J.G. Ojemann. 2005. Electrocorticography- based brain computer interface—the Seattle experience. IEEE Transactions on Neural Systems and Rehabilitation Engineering 14(2):194–198. Leuthardt, E.C., G. Schalk, D. Moran, and J.G. Ojemann. 2006a. The emerging world of motor neu - roprosthetics: a neurosurgical perspective. Neurosurgery 59(1):1–14; discussion 1–14. Leuthardt, E.C., K.J. Miller, G. Schalk, R.P. Rao, and J.G. Ojemann. 2006b. Electrocorticography- based brain computer interface—the Seattle experience. IEEE Transactions on Neural Systems and Rehabilitation Engineering 14(2):194–198. Leuthardt, E.C., G. Schalk, J. Roland, A. Rouse, and D.W. Moran. 2009. Evolution of brain-computer interfaces: Going beyond classic motor physiology. Neurosurgical Focus 27(1):E4. Leuthardt, E.C., C. Gaona, M. Sharma, N. Szrama, J. Roland, Z. Freudenberg, J. Solis, J. Breshears, and G. Schalk. 2011. Using the electrocorticographic speech network to control a brain-computer interface in humans. Journal of Neural Engineering 8(3):036004. Levine, S.P., J.E. Huggins, S.L. BeMent, R.K. Kushwaha, L.A. Schuh, E.A. Passaro, M.M. Rohde, and D.A. Ross. 1999. Identification of electrocorticogram patterns as the basis for a direct brain interface. Journal of Clinical Neurophysiology 16(5):439–447. Levine, S.P., J.E. Huggins, S.L. BeMent, R.K. Kushwaha, L.A. Schuh, M.M. Rohde, E.A. Passaro, D. A. Ross, K.V. Elisevich, and B.J. Smith. 2000. A direct brain interface based on event-related potentials. IEEE Transactions on Neural Systems Rehabilitation Engineering 8(2):180–185. Loeb, G.E., A.E. Walker, S. Uematsu, and B.W. Konigsmark. 1977. Histological reaction to vari - ous conductive and dielectric films chronically implanted in the subdural space. Journal of Biomedical Materials Research 11(2):195–210. Margalit, E., J.D. Weiland, R.E. Clatterbuck, G.Y. Fujii, M. Maia, M. Tameesh, G. Torres, S.A. D’Anna, S. Desai, D.V. Piyathaisere, A. Olivi, E. de Juan, Jr., and M.S. Humayun. 2003. Visual and electrical evoked response recorded from subdural electrodes implanted above the visual cortex in normal dogs under two methods of anesthesia. Journal of Neuroscience Methods 123(2):129–137.

OCR for page 123
134 FRONTIERS OF ENGINEERING McFarland, D.J., G.W. Neat, and J.R. Wolpaw. 1993. An EEG-based method for graded cursor control. Psychobiology 21:77–81. McFarland, D., D. Krusienski, W. Sarnacki, and J. Wolpaw. 2008a. Emulation of computer mouse con- trol with a noninvasive brain-computer interface. Journal of Neural Engineering 5(2):101–110. McFarland, D.J., W. Sarnacki, and J.R. Wolpaw. 2008b. Electroencephalographic (EEG) control of three-dimensional movement. Paper presented at the Society for Neuroscience Annual Meeting, Washington, DC, November 15–19. Millan, J.R., F. Renkens, J. Mourino, and W. Gerstner. 2004. Noninvasive brain-actuated control of a mobile robot by human EEG. IEEE Transactions on Biomedical Engineering 51(6):1026–1033. Moran, D.W., and A.B. Schwartz. 1999a. Motor cortical representation of speed and direction during reaching. Journal of Neurophysiology 82(5):2676–2692. Moran, D.W., and A.B. Schwartz. 1999b. Motor cortical activity during drawing movements: Popula - tion representation during spiral tracing. Journal of Neurophysiology 82(5):2693–2704. Moran, D.W., J.R. Wolpaw, and E.C. Leuthardt. 2008. Two-dimensional movement control using electrocorticographic signals in humans. Journal of Neural Engineering 5(1):75–84. Muller, K.R., M. Tangermann, G. Dornhege, M. Krauledat, G. Curio, and B. Blankertz. 2008. Machine learning for real-time single-trial EEG-analysis: From brain-computer interfacing to mental state monitoring. Journal of Neuroscience Methods 167(1):82–90. Nijboer, F., E.W. Sellers, J. Mellinger, M.A. Jordan, T. Matuz, A. Furdea, S. Halder, U. Mochty, D.J. Krusienski, T.M. Vaughan, J.R. Wolpaw, N. Birbaumer, and A. Kubler. 2008. A P300-based brain-computer interface for people with amyotrophic lateral sclerosis. Clinical Neurophysiology 119(8):1909–1916. Pfurtscheller, G., D. Flotzinger, and J. Kalcher. 1993. Brain-computer interface—a new communi- cation device for handicapped persons. Journal of Microcomputer Applications 16:293–299. Pfurtscheller, G., C. Guger, G. Mueller, G. Krausz, and C. Neuper. 2000. Brain oscillations control hand orthosis in a tetraplegic. Neuroscience Letters 292(3):211–214. Pfurtscheller, G., B. Graimann, J.E. Huggins, S.P. Levine, and L.A. Schuh. 2003. Spatiotemporal patterns of beta desynchronization and gamma synchronization in corticographic data during self-paced movement. Clinical Neurophysiology 114(7):1226–1236. Pistohl, T., T. Ball, A. Schulze-Bonhage, A. Aertsen, and C. Mehring. 2008. Prediction of arm movement trajectories from ECoG-recordings in humans. Journal of Neuroscience Methods 167(1):105–114. Polikov, V.S., P.A. Tresco, and W.M. Reichert. 2005. Response of brain tissue to chronically implanted neural electrodes. Journal of Neuroscience Methods 148(1):1–18. Ramsey, N.F., M.P. van de Heuvel, K.H. Kho, and F.S. Leijten. 2006. Towards human BCI appli- cations based on cognitive brain systems: An investigation of neural signals recorded from the dorsolateral prefrontal cortex. IEEE Transactions on Neural Systems and Rehabilitation Engineering 14(2):214–217. Rohde, M.M., S.L. BeMent, J.E. Huggins, S.P. Levine, R.K. Kushwaha, and L.A. Schuh. 2002. Quality estimation of subdurally recorded, event-related potentials based on signal-to-noise ratio. IEEE Transactions on Biomedical Engineering 49(1):31–40. Sanchez, J.C., A. Gunduz, P.R. Carney, and J.C. Principe. 2008. Extraction and localization of me - soscopic motor control signals for human ECoG neuroprosthetics. Journal of Neuroscience Methods 167(1):63–81. Schalk, G., E.C. Leuthardt, D. Moran, J. Ojemann, and J.R. Wolpaw. 2004a. Two-dimensional cursor control using electrocorticographic signals in humans. Proceedings of the Society for Neurosci- ence Annual Meeting, San Diego, Calif., October 23–27, 2004. Schalk, G., D.J. McFarland, T. Hinterberger, N. Birbaumer, and J.R. Wolpaw. 2004b. BCI2000: A general-purpose brain-computer interface (BCI) system. IEEE Transactions on Biomedical Engineering 51(6):1034–1043.

OCR for page 123
135 THE EVOLUTION OF BRAIN-COMPUTER INTERFACES Schalk, G., J. Kubánek, K.J. Miller, N.R. Anderson, E.C. Leuthardt, J.G. Ojemann, D. Limbrick, D. Moran, L.A. Gerhardt, and J.R. Wolpaw. 2007. Decoding two-dimensional movement trajectories using electrocorticographic signals in humans. Journal of Neural Engineering 4(3):264–275. Schalk, G., K.J. Miller, N.R. Anderson, J.A. Wilson, M.D. Smyth, J.G. Ojemann, D.W. Moran, J.R. Wolpaw, and E.C. Leuthardt. 2008. Two-dimensional movement control using electro- corticographic signals in humans. Journal of Neuroscience Methods 5(1):75–84. Serruya, M.D., N.G. Hatsopoulos, L. Paninski, M.R. Fellows, and J.P. Donoghue. 2002. Instant neural control of a movement signal. Nature 416(6877):141–142. Seymour, J.P., and D.R. Kipke. 2007. Neural probe design for reduced tissue encapsulation in CNS. Biomaterials 28(25):3594–3607. Spataro, L., J. Dilgen, S. Retterer, A.J. Spence, M. Isaacson, J.N. Turner, and W. Shain. 2005. Dexamethasone treatment reduces astroglia responses to inserted neuroprosthetic devices in rat neocortex. Experimental Neurology 194(2):289–300. Srinivasan, R., P.L. Nunez, and R.B. Silberstein. 1998. Spatial filtering and neocortical dynamics: Estimates of EEG coherence. IEEE Transactions on Biomedical Engineering 45:814–826. Sutter, E.E. 1992. The brain response interface: Communication through visually-induced electrical brain responses. Journal of Microcomputer Applications 15:31–45. Szarowski, D.H., M.D. Andersen, S. Retterer, A.J. Spence, M. Isaacson, H.G. Craighead, J.N. Turner, and W. Shain. 2003. Brain responses to micro-machined silicon devices. Brain Research 983(1-2):23–35. Taylor, D.M., S.I. Tillery, and A.B. Schwartz. 2002. Direct cortical control of 3D neuroprosthetic devices. Science 296(5574):1829–1832. Vaughan, T.M., D.J. McFarland, G. Schalk, W.A. Sarnacki, D.J. Krusienski, E.W. Sellers, and J.R. Wolpaw. 2006. The Wadsworth BCI Research and Development Program: At home with BCI . IEEE Transactions on Neural Systems and Rehabilitation Engineering 14(2):229–233. Velliste, M., S. Perel, M.C. Spalding, A.S. Whitford, and A.B. Schwartz. 2008. Cortical control of a prosthetic arm for self-feeding. Nature 453(7198):1098–1101. Vidal, J.J. 1977. Real-time detection of brain events in EEG. IEEE Proceedings. Special Issue on Biological Signal Processing and Analysis 65:633–664. Vossler, D., M. Doherty, R. Goodman, L. Hirsch, J. Young, and D. Kraemer. 2004. Early safety experience with a fully implanted intracranial responsive neurostimulator for epilepsy. Paper presented at the Annual Meeting of the American Epilepsy Society (AES). New Orleans, La., December 3–7, 2004. Wang, W., S.S. Chan, D.A. Heldman, and D.W. Moran. 2007. Motor cortical representation of position and velocity during reaching. Journal of Neurophysiology 97(6):4258–4270. Williams, J.C., J.A. Hippensteel, J. Dilgen, W. Shain, and D.R. Kipke. 2007. Complex impedance spectroscopy for monitoring tissue responses to inserted neural implants . Journal of Neural Engineering 4(4):410–423. Wolpaw, J.R., and D.J. McFarland. 1994. Multichannel EEG-based brain-computer communication . Clinical Neurophysioogy 90(6):444–449. Wolpaw, J.R., and D.J. McFarland. 2004. Control of a two-dimensional movement signal by a nonin - vasive brain-computer interface in humans. Proceedings of the National Academies of the USA 101(51):17849–17854. Wolpaw, J.R., D.J. McFarland, G.W. Neat, and C.A. Forneris. 1991. An EEG-based brain-computer in- terface for cursor control. Electroencephalography and Clinical Neurophysiology 78(3):252–259. Wolpaw, J.R., N. Birbaumer, W.J. Heetderks, D.J. McFarland, P.H. Peckham, G. Schalk, E. Donchin, L.A. Quatrano, C.J. Robinson, and T.M. Vaughan. 2000. Brain-computer interface technology: A review of the first international meeting. IEEE Transactions on Rehabilitation Engineering 8(2):164–173.

OCR for page 123
136 FRONTIERS OF ENGINEERING Wolpaw, J.R., N. Birbaumer, D.J. McFarland, G. Pfurtscheller, and T.M. Vaughan. 2002. Brain- computer interfaces for communication and control. Clinical Neurophysiology 113(6):767–791. Yuen, T.G., W.F. Agnew, and L.A. Bullara. 1987. Tissue response to potential neuroprosthetic materials implanted subdurally. Biomaterials 8(2):138–141.