Introduction of brain computer interface to neurologists

Do-Hyung Kim; Hong Gi Yeom; Minjung Kim; Seung Hwan Kim; Tae-Won Yang; Oh-Young Kwon; Young-Soo Kim

doi:10.14253/acn.2021.23.2.92

Abstract

A brain-computer interface (BCI) is a technology that acquires and analyzes electrical signals from the brain to control external devices. BCI technologies can generally be used to control a computer cursor, limb orthosis, or word processing. This technology can also be used as a neurological rehabilitation tool for people with poor motor control. We reviewed historical attempts and methods toward predicting arm movements using brain waves. In addition, representative studies of minimally invasive and noninvasive BCI were summarized.

INTRODUCTION

Hans Berger first succeeded in measuring electrical signals generated in the human brain from the scalp in 1924 using an electroencephalogram (EEG), which provided useful information for clinical practice such as determining convulsions, sleep, and brain death. Some scientists have considered EEGs to be helpful for people who were physically disabled by controlling a computer or machine through appropriate signal processing (Fig. 1). These ideas have now been developed and supported by studies, and implemented as a brain-computer interface (BCI), brain-machine interface (BMI), neural control interface, mind-machine interface, and direct neural interface, with BCI and BMI being the most common.

Jacques Vidal introduced the name BCI. In a paper published in 1973, he defined BCI based on the ability to manipulate objects through brain waves.1 BMI was developed by a group of neurophysiologists who suspected that the imprecision of noninvasive EEG signals only made them useful for selecting from among a small number of options. They suggested that collecting electrical signals directly from the brain through invasive methods would allow the more-accurate control of external machines, which was a technique they named BMI.2 The recent developments of BCI and BMI have resulted in them being used interchangeably, and BMI can also be called intracortical BCI (iBCI).3

This article reviews the history and principles of iBCI research with a focus on research that predicts arm movements by analyzing the electrical activity in the brain, and minimally invasive and noninvasive BCI research that has attempted to overcome the limitations of iBCI. We aimed to explain these in a manner that would be easy for neurologists to understand.

HISTORY AND PRINCIPLES OF ARM-MOVEMENT PREDICTION

Initial study

Attempts to analyze movements by deciphering electrical signals in the cerebral cortex of monkeys began in the 1960s. Evarts conducted experiments using monkeys trained to receive fruit juice as a reward for performing wrist flexion and extension.4 Of the pyramidal tract neurons in the precentral gyrus of monkeys, 31 were identified as being involved in wrist movement, and the spike frequency (firing rate) of the neurons increased or decreased when flexion and extension were performed. Spike frequency was also indicated to increase when loads exerted forces during the movements.

Humphrey et al.5 subsequently conducted experiments on monkeys using the same methodology as Evarts, and measured the spike frequency (firing rates) of pyramidal tract neurons in the precentral gyrus during flexion and extension. Humphrey et al.5 aimed to predict wrist position by entering their data into the following equation:

Φ (t) = a_{0} + \sum_{i}^{} a_{i} + U_{i} (t)

where t is the time, Φ(t) is the wrist position at that time, and Ui (t) is the spike frequency at the ith unit. When performing flexion and extension, the spike frequency and wrist position information were repeatedly measured at certain time increments and entered into the above formula to obtain the constants a₀ and a_i. This indicated that arm position can be inferred from the spike frequency alone.

Direction prediction in two dimensions

The most impressive study on predicting the arm movements of monkeys by analyzing electrical activity generated in the cerebral cortex was reported by Georgopoulos et al.6 in 1982. Their experiment used a monkey that was trained to move its arm in the direction of table sections randomly lit by nine light emitting diodes (LEDs) (0, 45, 90, 135, 180, 225, 270, and 315 degrees, and center). When the neurons in the motor cortex area of the monkey responsible for arm movements were measured, certain neurons exhibit increased spike frequencies when the arm was moved in a specific direction, which decreased when moving in the opposite direction (Fig. 2). Georgopoulos et al.6 measured the spike frequency of a particular neuron as the arm moved in each direction, and tried to determine the preferred movement direction for the neuron using a set of formulas. The first was

y_{i} = b_{0} + b_{1} \sin θ_{i} + b_{2} \cos θ_{i}

where y_i (i = 1,2, 3, ..., 8) is the average spike frequency that occurs when moving in the θ_i direction (θ_i = 0, 45, 90, …, 315 degrees). Repeatedly entering the average spike frequencies measured in each direction will yield the values of constants b₀, b₁, and b₂. The following formula can infer the spike frequency of neurons when moving in each direction:

D = b_{0} + b_{1} \sin θ + b_{2} \cos θ

This formula can be converted to

D = b_{0} + c_{1} \cos (θ - θ_{0})

where θ₀ is the preferred direction and b₀, b₁, and b₂ are the regression coefficients. This formula can be used to produce a plot with the angle at which the LED is lit on the x-axis and the spike frequency on the y-axis. The angle corresponding to the vertex of this graph is the direction in which the neuron prefers to move the arm of the monkey.

Direction prediction in three dimensions, and artificial intelligence

Georgopoulos et al.7 repeated the same experiment in a three-dimensional cube environment rather than a two-dimensional plane, since neurons with directionality exist in a three-dimensional environment, and found that the direction in which the neurons prefer to move the arm can be determine. They subsequently demonstrated that it is possible to calculate the direction of the neuronal population using a formula that gives weight if the spike frequency is high during an analysis, by also collecting data from adjacent neurons.8

With the recent development of artificial intelligence technology, the flexion and extension of the hand of a monkey can be predicted after only 5-10 minutes of training.9 A previous study indicated that it is possible to distinguish the direction by analyzing the local field potential instead of just the measured spike.10

HISTORY OF THE IBCI

Animal study

The most successful study on iBCI using monkeys was reported in 2008 by Velliste and colleagues, including Andrew Schwartz who had worked with Georgopoulos. They successfully moved a robotic arm in a three-dimensional space using the brain wave signals of a monkey.11 Electrical signals from the brain were first obtained by inserting intracortical microelectrode arrays into the primary motor cortex area of the monkey. They then identified channels that were preferentially involved during movements along the x-axis, y-axis, and z-axis in a three-dimensional space. By calculating the spike frequency in the corresponding channel, they attempted to move the robotic arm in a three-dimensional space to the target point so that the monkeys could eat food by bringing it to the mouth (Fig. 3). Two monkeys were included in the study, with accuracies of 67% and 78%.

Human study

The successful experiments on monkeys led to experiments involving humans. The first study to be performed on humans included two patients who had been quadriplegic for several years due to a brain stem infarction.12 Microelectrode arrays with 96 channels were inserted into the motor cortex hand areas of the dominant hemisphere of these patients. Similarly to the monkey experiment, the channels that preferentially moved the arm forward, backward, left and right, and up and down, and those that preferentially clenched and released the fist were identified, and the robotic arm could then be moved based on this analysis.

Another study was conducted on patients with spinocerebellar degeneration.13 In that study, two intracortical microelectrode arrays were inserted under the guidance of magnetic resonance imaging and functional magnetic resonance imaging. Unlike in the abovementioned study, the robotic arm could also be moved clockwise and counterclockwise in addition to clenching and releasing the fist by moving forward, backward, left and right, and up and down.

Beyond a single robotic arm

Another subsequent study was performed on a patient who developed quadriplegia from a cervical spine cord injury caused by a traffic collision.14 A microelectrode array was inserted into the left cerebral hemisphere hand area of the subject, and the subject wore a neuromuscular electrical stimulation sleeve on their right arm that moved their arm, instead of a robotic arm, so that the four movements of wrist flexion, wrist extension, and radial and ulnar deviation of the wrist were possible.

Studies on transmitting sensory information to the brain through a robotic arm have also been conducted.15 Two microelectrode arrays (with 88 channels) were inserted into the motor cortex hand and arm regions of patients with a cervical-spine cord injuries, and two microelectrode arrays (with 32 channels) were inserted into the hand and arm cutaneous regions of the somatosensory cortex. Torque measurements of the robotic arm were set to transmit the stimulus to the electrodes inserted into the somatosensory cortex, and a robotic arm moved and grabbed the target object. A comparison between the time to move and grab the target object was performed when the sensory stimulus was either delivered or not delivered, which indicated that providing sensory information shortened the required time.

MINIMALLY INVASIVE BCI

The technique of inserting microelectrodes into the brain carries significant risks, including causing infection or seizures. In addition, the signal measurement sensitivity of the electrode decreased with time.16 A proposed minimally invasive method as an efficient and safer option is currently being studied.

Stentrode

Studies have been conducted using a Stentrode (a stent device attached to electrodes that is capable of measuring electrical signals from the brain) with the help of intracranial endovascular management (Fig. 4A). An animal experiment was initially conducted. In this experiment, electrical activities of the brain were obtained by inserting a Stentrode into the superior sagittal sinus. Comparing the quality of the electrical activities collected from the brain indicated that the bandwidths of those collected through invasive methods (classic electrodes implant in subdural space and epidural space) were not as good as those collected through subdural electrodes, but were similar to those collected through epidural electrodes. However, the power obtained was indicated to be similar to that for the subdural electrode.17

A subsequent experiment was conducted on amyotrophic lateral sclerosis (ALS) patients. In that study, the cursor of a computer mouse was moved by pupil movement, and left clicking or zooming was performed based on information obtained from the Stentrode.18 However, the Stentrode did not appear to be permanently usable, and its applications may be restricted by certain risks.

Neuralink

Meanwhile, Elon Musk has also been interested in a minimally invasive BCI, and has conducted several studies on the subject. Musk19 founded a company called Neuralink, developed small and flexible electrode “threads” (with up to 3,072 electrodes per array distributed across 96 neural threads), and conducted animal experiments (Fig. 4B). When conducting experiments on mice in 2019, the Link tool that covers neural “threads” and transmits EEG information was inserted under the skin. In 2020, Link was successfully upgraded to wirelessly transmit the EEG information of pigs through neural threads. In 2021, it was announced on YouTube that Neuralink succeeded in playing the “mindpong” game with neural activity by inserting a device into the monkey. However, the findings of these studies need further verification.

NONINVASIVE BCI

Since the majority of noninvasive BCI applications use scalp EEGs, the information obtained from brain signals is minimal, and sophisticated movements such as arm movements cannot be implemented in the above-mentioned iBCI and minimally invasive BCI studies. However, compared with invasive methods, several noninvasive BCI-based methods have been devised for quadriplegic patients due to their minimal risk, and have the advantage of being able to permanently retain brain waves.

Visual evoked potentials

The first study on noninvasive BCIs was attempted by Vidal. The experiment was performed by moving a cursor on a computer screen using visual evoked potentials (VEPs) to escape a maze.20 However, it was limited by a lack of precision.

Slow cortical potentials

When concentration or movement occurs, EEG potentials change after 0.5-10 seconds, which is called slow cortical potentials (SCPs). Increases and decreases in cerebral activity can be observed as negative and positive SCPs, respectively.21 Using the SCP phenomenon, ALS patients succeeded in delivering a message by selecting letters by moving the cursor on the computer screen.22 However, the signal appears slowly and an accurate distinction is not made, which restricts applications of this technique.

Event-related desynchronization and synchronization

Waveforms at 8-12 Hz observed in the motor cortex area of the cerebrum are called mu rhythms, and are known to be related to movement. Decreases in mu and beta rhythms are observed in the motor cortex area during movement preparation and execution phases, which is called event-related desynchronization (ERD).23 Increases in mu and beta rhythms are observed in the motor cortex area before movement ceases, which is called event-related synchronization (ERS). However, BCI research using ERD and ERS has been restricted by their low recognition rates.

P300 wave

Applying a stimulus induces a positive peak in the parietal lobe that appears about 300 ms later, which is called P300 and is used in noninvasive BCI. The most representative method involves using a 6 × 6 screen containing letters and several word commands.24 If the subject continues to look at the desired letter while the lights in the rows and columns flash randomly, a P300 wave occurs when the lights of the desired letter are turned on. Using this method, the text input of the subject is a noninvasive BCI using the P300 VEP. This method has been established as the most basic BCI method utilizing P300 for applications in future studies, but it has the disadvantages of needing visual stimulation and continuous flashes of light.

Steady-state VEP

Noninvasive BCI also includes methods based on steadystate VEPs.25 This method utilizes a phenomenon in which the intensities at certain frequencies in the occipital lobe increase in proportion to the visual stimulus frequencies. For example, a visual stimulus blinking at 7 Hz is proportionally measured at 7 Hz by the EEG on the occipital lobe, which may be identified by neurologists as “photic stimulation” during an EEG examination. Although its relatively good recognition rate is an advantage, it also has the disadvantage of requiring visual stimulation and inconvenience from continuously seeing the blinking objects. It may also cause headaches or convulsions in some cases.

CONCLUSION

This article has introduced and explained the history of brain wave acquisition and processing during arm movements, adaptation to iBCI, and various minimally invasive BCI and noninvasive BCI applications in a way that neurologists can easily understand. iBCI has the advantage of predicting arm movements and being able to make precise movements, but it can cause infection or seizures, and is difficult to use permanently due to the signal measurement sensitivity of the electrode decreasing over time.16 While BCI is minimally invasive compared to iBCI, it lacks precision, and the signal measurement sensitivity of the electrode decreases over time.17 The noninvasive method is safer and is capable of continuously receiving EEG signals, but it has difficulty in implementing precise movements such as in the arm, and it therefore remains restricted to selecting from among only several options. It also often has a poor recognition rate or needs additional stimuli such as flashlights.

Each method has several advantages and disadvantages, and so none can be concluded as being better than the others. However, since these technologies can help quadriplegic patients suffering from conditions such as ALS, brain stem stroke, and cervical spine cord injury to manipulate the external world as they desire, interest in these techniques might particularly come from neurologists who directly interact with them. Since the currently developed technology is difficult to implement outside the laboratory due to its high cost, a cost-effective implementation strategy is needed to enable practical applications that are helpful in daily life.

Notes

Conflicts of Interest

The authors have no conflicts to disclose.

REFERENCES

1. Vidal JJ. Toward direct brain-computer communication. Annu Rev Biophys Bioeng. 1973; 2:157–180.

2. Nicolelis M. Beyond boundaries: The new neuroscience of connecting brains with machines---and how it will change our lives. 1st ed. New York: Times Books;2011. p. 1–353.

3. Rastogi A, Vargas-Irwin CE, Willett FR, Abreu J, Crowder DC, Murphy BA, et al. Neural representation of observed, imagined, and attempted grasping force in motor cortex of individuals with chronic tetraplegia. Sci Rep. 2020; 10:1429.

4. Evarts EV. Relation of pyramidal tract activity to force exerted during voluntary movement. J Neurophysiol. 1968; 31:14–27.

5. Humphrey DR, Schmidt EM, Thompson WD. Predicting measures of motor performance from multiple cortical spike trains. Science. 1970; 170:758–762.

6. Georgopoulos AP, Kalaska JF, Caminiti R, Massey JT. On the relations between the direction of two-dimensional arm movements and cell discharge in primate motor cortex. J Neurosci. 1982; 2:1527–1537.

7. Georgopoulos AP, Schwartz AB, Kettner RE. Neuronal population coding of movement direction. Science. 1986; 233:1416–1419.

8. Georgopoulos AP, Kettner RE, Schwartz AB. Primate motor cortex and free arm movements to visual targets in three-dimensional space. II. Coding of the direction of movement by a neuronal population. J Neurosci. 1988; 8:2928–2937.

9. Wessberg J, Stambaugh CR, Kralik JD, Beck PD, Laubach M, Chapin JK, et al. Real-time prediction of hand trajectory by ensembles of cortical neurons in primates. Nature. 2000; 408:361–365.

10. Flint RD, Lindberg EW, Jordan LR, Miller LE, Slutzky MW. Accurate decoding of reaching movements from field potentials in the absence of spikes. J Neural Eng. 2012; 9:046006.

11. Velliste M, Perel S, Spalding MC, Whitford AS, Schwartz AB. Cortical control of a prosthetic arm for self-feeding. Nature. 2008; 453:1098–1101.

12. Hochberg LR, Bacher D, Jarosiewicz B, Masse NY, Simeral JD, Vogel J, et al. Reach and grasp by people with tetraplegia using a neurally controlled robotic arm. Nature. 2012; 485:372–375.

13. Collinger JL, Wodlinger B, Downey JE, Wang W, Tyler-Kabara EC, Weber DJ, et al. High-performance neuroprosthetic control by an individual with tetraplegia. Lancet. 2013; 381:557–564.

14. Bouton CE, Shaikhouni A, Annetta NV, Bockbrader MA, Friedenberg DA, Nielson DM, et al. Restoring cortical control of functional movement in a human with quadriplegia. Nature. 2016; 533:247–250.

15. Flesher SN, Downey JE, Weiss JM, Hughes CL, Herrera AJ, Tyler-Kabara EC, et al. A brain-computer interface that evokes tactile sensations improves robotic arm control. Science. 2021; 372:831–836.

16. Polikov VS, Tresco PA, Reichert WM. Response of brain tissue to chronically implanted neural electrodes. J Neurosci Methods. 2005; 148:1–18.

17. Oxley TJ, Opie NL, John SE, Rind GS, Ronayne SM, Wheeler TL, et al. Minimally invasive endovascular stent-electrode array for high-fidelity, chronic recordings of cortical neural activity. Nat Biotechnol. 2016; 34:320–327.

18. Oxley TJ, Yoo PE, Rind GS, Ronayne SM, Lee CMS, Bird C, et al. Motor neuroprosthesis implanted with neurointerventional surgery improves capacity for activities of daily living tasks in severe paralysis: first in-human experience. J Neurointerv Surg. 2021; 13:102–108.

19. Musk E. An integrated brain-machine interface platform with thousands of channels. J Med Internet Res. 2019; 21:e16194.

20. Vidal JJ. Real-time detection of brain events in EEG. Proc IEEE. 1977; 65:633–641.

21. Wolpaw JR, Birbaumer N, McFarland DJ, Pfurtscheller G, Vaughan TM. Brain-computer interfaces for communication and control. Clin Neurophysiol. 2002; 113:767–791.

22. Birbaumer N, Ghanayim N, Hinterberger T, Iversen I, Kotchoubey B, Kübler A, et al. A spelling device for the paralysed. Nature. 1999; 398:297–298.

23. Pfurtscheller G, Lopes da Silva FH. Event-related EEG/MEG synchronization and desynchronization: basic principles. Clin Neurophysiol. 1999; 110:1842–1857.

24. Farwell LA, Donchin E. Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. Electroencephalogr Clin Neurophysiol. 1988; 70:510–523.

25. Wang Y, Gao X, Hong B, Jia C, Gao S. Brain-computer interfaces based on visual evoked potentials. IEEE Eng Med Biol Mag. 2008; 27:64–71.

Fig. 1.

Concept image of a brain-computer interface. Figure adapted from Wolpaw et al.21

Fig. 2.

Direction predictions based on neuron firing rates. Impulse activity was recorded from a single neuron during repeated arm movements toward the target. Neurons fire just (about 300 ms) before the onset of movement (i.e., 0 ms). Each dot represents a neuron firing, and each row on the vertical axis represents a repetitive arm movement in the same direction. Firing rates increased when the arm was moved 180 degrees in the desired direction and decreased when the arm was moved back to 0 degrees, suggesting that the recorded neuron prefers to move the arm toward 180 degrees.

Fig. 3.

A monkey controlling a robotic arm with its brain waves. Figure adapted from Velliste et al.11

Fig. 4.

Minimally invasive devices. (A) Stentrode with 8 × 750 μm electrode discs self-expanding during administration by a 4F catheter. Scale bar, 3 mm. The device is usually implanted in the superior sagittal sinus to obtain the electrical activity of the brain. Figure adapted from Oxley et al.17 (B) Neural threads. “Tree” probes with electrode contacts spaced by 75 μm. Scale bar, 100 μm. Figure adapted from Musk.19