Can Brain Control Guide or Refine Limb Control?
TASK GROUP DESCRIPTION
Decoding Brain Ensemble Signals
The possibility of brain control of artificial limbs became a realistic prospect when a population of brain signals recorded in the motor cortex of behaving monkeys was successfully decoded to provide accurate information about motor parameters (Humphrey et al., Science 170:758-762, 1970) and the direction of movement in space (Georgopoulos et al., Experimental Brain Research 7(Suppl.):327-336, 1983), even before the onset of movement. The first successful prediction of a complete, upcoming 3-D reaching movement trajectory was achieved shortly thereafter (Georgopoulos et al., Journal of Neuroscience 8:2928-2937, 1988).
Implanted Electrodes for Prosthetic Control
The application of this discovery for brain-controlled motor prostheses requires the chronic implantation of at least tens of recording microelectrodes inside the brain. To that end the main challenge in the 1980s and 1990s was to develop microelectrodes suitable for chronic implantation, such that they (a) would be made of material safe for the brain, (b) would
maximize the number of recording sites per unit of electrode area (to limit the total number of implanted electrodes), and (c) would be associated with appropriate microelectronics to ensure signal amplification and preprocessing in close proximity to the recording site. Significant progress in all these three domains led during the past few years to a flurry of testing applications of neuroprosthetic control. Several laboratories are testing different kinds of systems of microelectrodes implanted in the motor and parietal cortices of monkeys and their capability for prosthetic control (for reviews see Taylor et al., 2002; Schwartz, 2004), and such applications on human subjects are also underway (Hochberg et al., 2006). However, major concerns remain. For example, the long-term (e.g., years) safety of implanted microelectrodes on the brain is essentially unknown, both with respect to the possible toxicity of the materials the electrodes are made of and the possible damage to the brain given the brain motion relative to the electrodes. However, the successful long-term safety and experience (>10 years) of much larger deep-brain-stimulating electrodes, used for the treatment of movement disorders, such as Parkinson’s disease, suggests that the brain can tolerate at least some types of electrodes quite well. In addition, the length of time for which implanted electrodes will continue to provide good-quality signals for prosthetic control is also to be determined.
Noninvasive Brain Signals for Prosthetic Control
Ideally, it would be best to use brain signals recorded by noninvasive ways to control prosthetic devices. Substantial work has been done on using electroencephalographic (EEG) signals for the purpose of direct communication of the brain with the environment, for example, by moving a cursor on a computer screen. The use of EEG signals for prosthetic control would be a major advance and would bypass most of the safety and other concerns associated with implanted electrodes. In addition, since EEG reflects brain activity from many areas, such signals possess the potential of being useful in other applications, such as brain-aided cognitive therapy, rehabilitation, remediation, and biofeedback. Recent studies demonstrated the power of magnetoencephalographic (MEG) signals for predicting upcoming moment trajectories (Georgopoulos et al., 2005), using the same decoding methods as originally described (see above). Since MEG is noninvasive and complementary to EEG, these findings suggest that EEG would also be a good predictor, as indeed has been found in preliminary studies (Georgopoulos et al., 2005). Nevertheless, noninvasive studies can-
not capture the single neuron scale of measurement as intracerebral microelectrode arrays can, and the limits to such noninvasive information extraction have not been determined.
Initial Challenges to Consider
In the intact animal, signals from the motor cortex are directed to the spinal cord (from layer 5 pyramidal cells) as well as to other cortical and subcortical areas, all parts of a distributed dynamic motor control network. Ideally, the basic functionality of such a network should be incorporated in a prosthetic limb, in order for the full benefit and impact of using motor cortical signals to be achieved. This would be particularly useful for the simultaneous or temporally overlapping control of multiple aspects of limb motor function, including hand trajectory in space, opening, closing, or shaping of the hand, force intensity, etc. A first step in that direction might be to implement spinal-cord-like circuitry in the prosthetic limb (Georgopoulos et al., Science 237:301, 1987). The use of brain signals from multiple brain areas recorded from simultaneously (e.g., parietal cortex, cerebellum, basal ganglia) or extracted from noninvasive brain signals would provide integrating information from a wider network. Alternatively, such signals and/or network principles could be incorporated in the prosthetic limb. In other words, the drive would be for a “smart” prosthetic limb with a spinal-like circuitry and additional neural network integration inspired from known interactions among brain areas. The unifying point is to use the brain of the prosthetic to fill in the computation that the central nervous system naturally provides when motor intention is generated. It is very possible that such a smart prosthetic limb would be much more amenable for efficient and effective control by brain signals of various motor functions.
Georgopoulos, A. P., F. J. Langheim, A. C. Leuthold, and A. N. Merkle. 2005. Magnetoencephalographic signals predict movement trajectory in space. Experimental Brain Research 167:132-135.
Hochberg, L. R., J. D. Serruya, G. M. Friehs, J. A. Mukand, M. Saleh, A. H. Caplan, A. Branner, D. Chen, R. D. Penn, and J. P. Donoghue. 2006. Neuronal ensemble control of prosthetic devices by a human with tetraplegia. Nature 442:164-171.
Schwartz, A. B. 2004. Cortical neural prosthetics. Annual Review of Neuroscience 27: 487-507.
Taylor, D. M., T. Helms, I. Stephen, and A. B. Schwartz. 2002. Direct cortical control of 3D neuroprosthetic devices. Science 296:1829-1832.
TASK GROUP SUMMARY
Summary written by:
Kirk Fernandes, Graduate Student/Journalist, Center for Science and Medical Journalism, Boston University
Task group members:
Richard Andersen, Professor, Biology, Caltech
Jose Carmena, Assistant Professor, Electrical Engineering and Computer Sciences, University of California, Berkeley
John Donoghue, Director, Brain Science Program, Neuroscience Department, Brown University
Alexander Dromerick, Professor, Rehabilitation Medicine and Neurology Department, National Rehabilitation Hospital/Georgetown University
Leon Esterowitz, Program Director, Bioengineering and Environmental Systems, National Science Foundation
Kirk Fernandes, Graduate Student/Journalist, Center for Science and Medical Journalism, Boston University
Apostolos Georgopoulos, Regents Professor, Director, The Domenici Research Center for Mental Illness Department, University of Minnesota Medical School
Simon Giszter, Associate Professor, Neurobiology and Anatomy, Drexel University College of Medicine
Selcuk Guceri, Dean, College of Engineering, Drexel University
Jiping He, Professor and Director, Bioengineering and Center for Neural Interface Design, Arizona State University
Leigh Hochberg, Instructor/Investigator, Neurology/Neuroscience Department, Harvard Medical School/Massachusetts General Hospital/ Brown University/VAMC
Robert Kirsch, Associate Professor, Biomedical Engineering, Case Western Reserve University
Zelma Kiss, Assistant Professor of Clinical Neurosciences, University of Calgary
Conrad Kufta, Director of Clinical Development, Innovative Neurotronics Inc.
Steven Schiff, Brush Chair Professor of Engineering, Center for Neural Engineering, The Pennsylvania State University
Arthur M. Sherwood, Science and Technology Advisor, National Institute on Disability and Rehabilitation Research (NIDRR), U.S. Department of Education
It seems that in life the most effortless actions can be the most challenging to artificially replicate. Consider the simple task of picking up and drinking a cup of water. In the simplest terms that task starts with an intention to pick up the cup. The intention sets off movement-related neurons to “fire” or “spike” in our brains. Signals travel through the nervous system, leaping across synapses between neurons, activating the final common path of motorneurons, and eventually communicating with the actuators (muscles) of the limb. The arm, hand, and fingers move toward the cup. The visual system observes and reports progress, as do sensory neurons in our fingertips, hands, and arms report information, such as position and velocity, back to our brain, which then sends additional signals to adjust movement accordingly based on our expectations. Eventually the cup has made it to our mouth and we take a sip, invoking another set of complex but effortless functions. The key to making effective smart prosthetics may lie in understanding the complex interactions originating in our brains.
An assemblage of 15 scientists, doctors, and agency representatives with backgrounds ranging from neuroscience to engineering welcomed the challenge during the 2006 Keck Futures Initiative Conference by tackling the question: Can brain control guide or refine limb control? “Brain control” refers to the process by which information about desired limb movement is recorded directly from neurons in a patient’s brain, decoded, and communicated to a smart prosthetic device that would then execute the desired movements. Following some discussion on the subject, the group submitted an answer to the query: “Yes”—as several participants have been involved in animal and human research that proves the basic concepts governing guidance via brain-machine interface. In moving ahead group members concentrated on the challenges of a brain-controlled smart prosthetic system, such as long-term stability, that when addressed, could ad-
vance the ability of the device to restore desired movement to a person with a physical disability.
In organizing the discussion one task group member suggested dividing the functionality of a brain-control system into three parts: command, communication, and control. Command refers to the implicit neuronal instructions generated by a patient’s intention to move a limb. The brain-machine command interface records and decodes that data in such a way that the information is both manageable and useful for the desired movement goals of the system.
The communication scheme then relays the data to the prosthetic device through existing biological pathways or artificial mechanisms in cases of paralysis. There could also be a two-way stream of information transfer, as the overall smart prosthetic system might benefit from the ability to feed data back to the brain.
Control refers to the system that actuates the desired movement and ensures that the desired movement occurs regardless of disturbances—most likely a computer interface built into a prosthetic device that senses key variables and makes needed adjustments.
Due in large part to the range of expertise of the group’s task members, the bulk of discussion focused on the command system. And in the end, participants took a holistic approach to addressing challenges, by creating a research roadmap for an ambitious goal-oriented initiative: develop a command interface that as a modular component in a smart prosthetic system, could fully restore function or have therapeutic value in the rehabilitation of lost movement.
The first issue task members needed to consider was the identification of the best targets in the brain for pertinent neuronal information. In other words, which specific neuronal signals, or combinations thereof, could be used to best determine the movement intent of the patient? Are the classical motor cortical areas ideal, or are the cognitive regions within the frontal and parietal cortices better sources for such information? The question sparked a lengthy debate over priorities when considering the most important variables of movement, such as trajectory, position, velocity, force, impedance, and posture. One participant noted that “pulling trajectories out of the brain is a piece of cake” and that velocity signals are “all over” in the motor cortex, but other variables were less studied. Other participants echoed concerns about a significant knowledge gap in this area, suggesting that it should be addressed in order to get a better handle on basic neurophysiology. They proposed a systematic research project that
would study basic movements in healthy individuals to determine which variables were critical to specific actions and what corresponding neurons in the brain could be targeted for data retrieval. Additional research initiatives could prioritize motor requirements for restoration. Once the various patterns are fully understood, one group member suggested the possibility of developing a generator or model that could simulate key brain signals. Such a device would allow for easier experimentation with communication and control interfaces.
Once neuronal targets are identified, scientists then need to consider the best way(s) to record that information. Current systems use an array of electrical sensors to monitor neuronal action or local field potentials, but the methods vary in degrees of invasiveness. To date, the most detailed information with the largest signal-to-noise ratio comes from microelectrodes that are inserted directly into the brain via a surgical procedure. Some task group members noted there are still many challenges with this approach, including the long-term potential for biomechanical failure demonstrated through ongoing projects. In fact, one researcher questioned whether “this is the way to go,” but another group member defended the method, saying, “There are many problems that remain to be resolved, but they are tractable.” In any case, the task group suggested future research endeavors explore the creation of probes with built-in stabilization features that would automatically adjust to brain movement within the skull. Other issues to consider with such microelectrodes would be the need to fully implant the device (perhaps using wireless communication systems), redundancy, risk minimization, and Magnetic Resonance Imaging (MRI) compatibility. (Implants would ideally be designed for MR compatibility, as heating and tissue injury could occur otherwise; however, people with Deep Brain Stimulation (DBS) systems can receive MRIs following careful protocols.)
Continuing on the same topic, the group moved to discussion of the less invasive method of reading EEG (electroencephalographic) signals. While current EEG skullcap devices do not require the insertion of electronics directly into the brain, the signal-to-noise ratio is much lower in EEG recordings than microelectrodes, providing less valuable information, according to some task group members. However, at least one researcher was optimistic that with improved technology, an EEG command interface could be used to control limb movement or perhaps be integrated with microelectrodes into a network of sensors that achieve ultimate movement goals.
A secondary issue to consider when examining methods for recording is whether the same apparatus can also be used to write information to sensory neurons in order to provide feedback information to the brain. One scientist pointed out that some amputees request transparent prosthetic hands to ensure they have visual feedback on the proximal relationship between their hand and the object with which it’s interacting. Such feedback would theoretically assist with adjusting and refining precise movements. One group member suggested that feedback signals could be processed in the prosthetic’s control mechanism, then simplified before communicated back to the brain or peripheral nerves. Ideally through training, the patient could learn how to interpret these signals.
Once scientists have the pertinent neuronal targets and the optimal recording devices, they can set forth to optimize the decoding process with ideally quick algorithms that allow for rapid communication of brain signals to the smart prosthetic. The group did not focus on such decoding algorithms, but stressed the importance of adaptation in mechanical and computational systems due to potential neuronal aberrations in altered brains. An adaptive decoding system would learn as the patient’s condition improves or worsens (in the case of ALS, for example). In addition, the group encouraged research that would support a co-adaptation strategy, in which both the command interface and control interface in the prosthetic would learn and refine movements together.
The group also put some attention toward considering the needs of disabled individuals when directing certain research initiatives. For example, one group member pointed out the fact that the great bulk of current brain-machine interface research focuses on arm movements despite the fact that the vast majority of amputation procedures are done on legs. And based on experiences with disabled individuals, other task group members stressed that ideal command, communication, and control systems should minimize attentional and training demands on the subject. Even public perception was a consideration, as the group wanted to dispel any general confusion over the phrase “brain control” and the false presumption that it describes a system that controls the brain rather than the prosthetic device.
Other issues that came under consideration included the appropriate representational frame for the prosthetic, the degrees of freedom required for commands extracted from the motor cortex, the required task effector, and whether movement specifications should be interpreted continuously, discretely, intermittently, or in some combination. Restoring complete
function to the arm and hand is a grand objective, but limitations in existing or even next-generation prostheses are unlikely to allow such performance to be delivered—even if the needed brain commands can be extracted. An alternative is a bottom-up approach that determines the needed properties of the brain computer interface (BCI) based on the movement goals of a particular prosthesis. Such design specifications would indicate what is “good enough” for a specific application, accelerating the near-term and effective use of BCIs in real applications. For example, it should be possible to determine how many independent command signals are really needed to control the movements used in the activities that are most important for daily function and realistically achievable. The importance of extracting kinematic-based movement commands (position and velocity) relative to kinetic variables (e.g., force) or stability-related quantities (e.g., impedance) would follow from these specifications in an application-specific manner. The specific variables required from the BCI are, in fact, determined entirely by the local control system of the prosthesis, in particular by local variables that are sensed and used as part of a feedback pathway—such feedback loops provide the language that the BCI must speak. The importance of direct motor commands (e.g., from primary motor cortex) relative to less direct “intention” commands from other brain areas will also depend upon the specific nature and sophistication of this local control system. Such a bottom-up approach would complement basic neurophysiological research by enabling effective use of emerging knowledge as the capability of prosthetic devices improves over time. Subsequently, such a BMI device and interface could provide the user an option and ability to take over or modify some of the local controls. This could allow for the incremental expansion of skills. The design of a smart prosthetic with appropriate initial capacities so as to support simple use from the brain and also provide the mechanical richness to allow skill development was considered to be a very desirable but complex objective.
The group also considered the idea that BMI systems could in the future be employed as part of a rehabilitation framework, not just as a substitution, in the spirit of refining movement. In such a framework the BMI could be designed to augment motor behavior in the short term in order to promote appropriate refinement and learning, but with the goal of the patient subsequently leaving the BMI behind, rather than using it as a substitute. Such a use would be particularly well suited to noninvasive systems, but might employ invasive BMI systems on occasion when clinically
warranted. The interesting and difficult issue in such rehabilitation applications is promoting initial function while avoiding ultimate dependence on the interface.
In the end, the task group set an ambitious goal for brain-controlled smart prosthetics: to restore the injured and afflicted to their prior social roles. But grand visions cannot be accomplished in giant leaps; instead, smaller tractable steps—starting with an increased understanding of basic neurophysiology—will build a solid foundation for smart limb prosthetics.