Human Performance Modification as a Function of Brain–Computer Interfaces
This chapter reviews recent progress in brain-computer interfaces (BCIs), including the potentially enabling role of nanotechnology. BCIs allow direct communication of neural signals with an external device. A large body of research concerns the ability to detect and translate neural activity, direct it to control a machine, and thereby affect human performance (Brunner et al., 2011). The most common applications are in rehabilitative medicine. For example, neural implants now enable a disabled patient to control a wheelchair, prosthesis, or voice simulator (Rebsamen et al., 2010; Bell et al., 2008; Brumberg and Guenther, 2010). Conversely, electronic signals may be used to stimulate portions of the brain to induce a particular motor response; this has been demonstrated only in animals (Arfin et al., 2009; Nuyujukian et al., 2011). All of the BCIs in current use appear to be very slow and do not perform as well as a healthy person can.
Several elements are required for a BCI to work. Perhaps the most important is that the brain signals are detected and understood. When a neuron fires, it emits a detectable electric signal. That is most commonly detected externally and noninvasively by electroencephalography (EEG), in which electrodes are placed on the scalp. EEG is widely used for detection of electrical activity associated with epileptic seizures or to characterize the overall status of electrical activity in the brain (Wolpaw et al., 2002). Researchers have made much progress in understanding which regions of the brain are activated during particular processes. Recent studies at the University of California, Berkeley have shown that it is possible to reconstruct sounds heard by a patient suffering from severe epilepsy on the basis of EEG spectra obtained from neural probes implanted in his brain (Pasley et al., 2012). The next step would be to extend the capability to analyze EEG spectra of thoughts and convert them to speech. This would be an exciting advance, in that it could allow people who have lost the ability to speak to recover it.
In the military realm, fighter-pilot helmets lined with EEG sensors have been developed as part of a cognitive avionics tool set (Schnell, 2009). The goal is to collect as much data as possible to measure pilot cognitive overloading and underloading in high-stress situations. However, the most important indicators of these conditions to date are eye-gaze and respiration data rather than EEG data. To make EEG data more useful, synchronization systems that can correlate EEG response to pilot expertise need to be developed. Research is continuing to extract higher-resolution and more useful information from the EEG data by using higher frequencies for better topographic resolution and by creating much denser arrays of electrodes; the latter is now possible with soft materials and advanced fabrication methods (Crone et al., 2006).
In addition to detecting brain activity externally, it is possible to implant devices directly into the brain. The advantage is that the electrodes are much closer to the source of electrical activity, so the signal-to-noise ratio is greatly improved and the localization provides a more direct idea of the origin of the activity (Vetter et al., 2003). A number of such devices have been developed (primarily in the United States), notably BRAINGATE, the Utah Probe, and the Michigan Probe (Archibald, 2005; Huys et al., 2011; Kipke et al., 2011). Such invasive probes or arrays of an EEG type of electrodes on the surface of the brain are contemplated only for patients who have severe disorders. The smallest probes currently available are much larger than individual neurons; serious damage is done to brain tissue on insertion, and the process is irreversible (He et al., 2006). It may be justified for severely disabled patients, but seems unlikely to be acceptable as an enhancement method in the near term. Major efforts are under way in a number of laboratories to integrate electrodes or meshes of electrodes with neurons (and other cell types) in culture, and this is potentially a less destructive approach (Lee et al., 2004). It has potential for repair of damaged neuronal tissue, including the possibility of using radiofrequency electronics so that signals can be read—or regions can be stimulated—remotely.
An alternative detection system is related more directly to physiology, particularly blood flow and glucose metabolism associated with activity in a particular region of the brain (Son and Yazici, 2005). That is the basis of functional magnetic resonance imaging (fMRI), positron-emission tomography (PET), and near-infrared functional (NIRF) imaging. Although fMRI and PET imaging are major imaging modalities in clinical medicine and brain research, neither can currently be implemented in a portable or even semiportable format (Son and Yazici, 2005).
NIRF imaging uses changes in the near-infrared absorption of deoxyhemoglobin vs. oxyhemoglobin to obtain information on changes in blood oxygenation (Cui et al., 2011; Hoshi, 2011). Blood oxygenation changes during high levels of activity, and substantial changes are observed in brain tissue. Arrays of near-infrared sources (for example, laser diodes) and detectors in combination can be used to generate low-resolution images of brain function. The penetration of near-infrared light through the skull of adults is modest (it is much higher in premature infants, and this technique is used in the neonatal intensive care unit).
Using brain signals to monitor cognition for use in training has demonstrated benefits. Some groups are pursuing augmented cognition whereby human performance is improved by designing for memorability. It is necessary for cognitive bottlenecks and firing patterns to be measured to detect cognitive states within some small amount of time (Son and Yazici, 2005). If that can be accomplished, perhaps behavior can be influenced.
Although a great deal of progress has been made in this realm, how the brain functions is still largely unknown (Son and Yazici, 2005; Hoshi, 2011). Even if neural events could be reliably correlated to the detection of sensory inputs or to the stimulation of specific motor activity, there is a fundamental issue of neuromuscular mismatch (Kirilina et al., 2012). The brain has evolved to control muscles, whereas BCI research strives to create interfaces between the brain and computers. Current neural implants are rather crude, and a more biological way to create an interface with the brain to enhance efficacy and decrease scarring is being sought and may be found in the next 20 years. The most promising direct applications of this research to human performance modification (HPM) are in developing therapies for existing disabilities. Even in those cases, the improvements do not approach the functionality of a healthy person. It is unlikely that human performance can be substantially enhanced via BCI in the near term.
According to Thomas Schnell of the University of Iowa, non-U.S. entities appear to be more active in such cognitive-function research. He believes that the international players that warrant attention are Israel, Germany, Japan, and the Netherlands. It should also be noted that Taiwan and South Korea have infrastructure and expertise in these fields; for example, the Korea Advanced Institute of Science and Technology has recently established a Department of Bio and Brain Engineering. There is also great interest in this research in China. The work that has been published in western peer-reviewed scientific journals suggests that China is still building expertise based on western innovation rather than pioneering novel research of its own. Nonetheless, China is pouring resources into this area.1
In the context of HPM, it is important to consider possibilities beyond physical modification of the human or the human's work environment. An intriguing example is that it may be possible to train a person to work with a machine or computer interface to enhance the person’s normal capabilities (Cui et al., 2012). A crude illustration of this concept can be seen in the case of a bicycle. A person is trained to create an interface with the bicycle to increase his or her ability to move. The person learns to move his or her legs in a specific way to move the pedals to produce propulsion, to balance his or her body on the machine, and to maneuver the machine through combinations of body movements and steering. A more advanced illustration might be the training of a person to manipulate machines through the coordinated movements and controlled thought patterns that are accessible with increasingly sensitive equipment. The concept would build on capabilities that are already in place, such as eye-movement measurements and bodytracking capabilities. An example of a nascent capability of this kind can be seen in the University of Washington application2 that uses a kinesthetic learning environment to teach students mathematics. The idea of training the user and the machine together may open a new frontier in HPM. Augmenting performance may not require as much if machine and user are trained together as it would otherwise.
NANOTECHNOLOGY FOR HUMAN PERFORMANCE MODIFICATION
Nanotechnology involves any application of materials that measure a few hundreds of nanometers or smaller. It can refer to a wide variety of technologies relevant to HPM, including electronics, microelectromechanical systems, energy harvesting and storage, and biomedicine. The field cuts across many of the other aspects of HPM in that it enables their implementation.
Advanced fabrication techniques allow ever-smaller computer chips, cameras, and antennae that can result in wireless devices that are small and light enough to be integrated into virtually every aspect of human endeavor. Smart phones, for instance, benefit from these advances, allowing more capabilities to be integrated in a device that is small and light enough to fit in a pocket. Devices such as smart phones arguably enhance human performance by making a vast amount of information accessible. The miniaturization is also enabling technologies that more directly alter human performance, such as proposed wearable augmented reality devices, including Sixth Sense—an interface based on gestures that is a predecessor of the type depicted in
1Thomas Schnell, Associate Professor, University of Iowa. Operator State Characterization Using Neurophysiological Measures. Presentation to the committee on March 8, 2012.
2For more information, see http://kinectmath.org/. Accessed August 2, 2012.
the film Minority Report—and Google Glasses, a heads-up display that promises to project smartphone functionality in the wearer’s field of view.3,4
Another aspect of nanotechnology that needs to be considered with regard to HPM is in the realm of biointerfaces. Nanoparticles can be smaller than cells, and this opens the possibility of using them to interact directly on a cellular level by inserting them into the body. A straightforward application is to enhance sensory perception via subdermal nanoparticles. Recently, Nokia filed a patent for a tattoo containing nanoparticles that vibrate when a cell-phone call is received.5 It is easy to imagine how such a consumer device can be applied to the battlefield; for instance, the tattoo could be coupled to chemical sensors to alert soldiers to the presence of toxic gases or explosives.
A more invasive use of nanotechnology for HPM is in the development of neural implants. These devices are implanted directly into the brain to detect electric signaling. To increase the signal-to-noise ratio and resolution, the probes need to be on the same scale as the neurons that they monitor, that is, a few micrometers. Furthermore, they must be made of materials that are biocompatible and produce little or no damage and scarring of the surrounding tissue. To achieve that, sophisticated nanomaterials are being developed.
Although much of the research in nanotechnology began in the United States, continued development for electronics and related applications is taking place globally, spurred largely by the consumer and military markets. U.S. superiority in the development of these and newer technologies cannot be assumed, especially given that a large fraction of the manufacturing of state-of-the-art electronics is taking place in Asia. The United States is a leader in basic research in neural engineering, but laboratories in Europe and in Asia (China in particular) are also active.
3For more information about Sixth Sense, see http://www.pranavmistry.com/projects/sixthsense. Accessed May 3, 2012.
4For more information about Google Glasses, see https://plus.google.com/u/0/111626127367496192147/posts. Accessed May 3, 2012.
5For more information, see https://docs.google.com/file/d/0B23n8sehUZyqckhLaG9qZ0hRSXFnQkctYXhDYTY4dw/edit?pli=1. Accessed June 27, 2012.