Researchers from the BrainGate consortium have, for the first time, shown that a brain–computer interface (BCI) can be used by people with paralysis to control a commercial tablet computer. A small device implanted in the motor cortex of the patient was able to interpret neural signals and translate these into mouse movement on a tablet.
For patients with paralysis, assistive technologies can be vital tools to regain functionality. Recent research has aimed to create a direct link between such assistive technology and the brain of a user. BCI-based devices have previously been used to operate a tablet for general programs such as web browsing, painting and games. However, the latest work with the team’s BrainGate2 sensors has demonstrated the ability to use an unmodified off-the-shelf tablet device. Participants were able to make use of commercial apps from the Google Play store, even when these were not included in the original trial design (PLOS ONE 10.1371/journal.pone.0204566).
Giving back control
The BCI uses a decoder to interpret brain waves for intended movement from the motor cortex and present these to the tablet in the same way as a signal from a Bluetooth mouse. Each participant had their own imagery to produce a click; for instance, one imagined clenching their left hand. When asked about how easy this method was to get to grips with, one of the patients said that “the tablet became second nature to me, very intuitive. It felt more natural than the times I remember using a mouse.”
During the study, three participants from the larger BrainGate2 clinical trial were asked to perform certain tasks, such as replying to e-mails and searching for videos and pictures of their hobbies. The patients were able to complete all tasks set for them and also demonstrated use of other apps of their choice, such as shopping or calculator apps. They were also monitored on the speed at which they could type when using the chat features. Participants were able to increase the speed at which they typed by making use of the word completion function, to achieve rates of up to 31 characters per minute.
One limitation of the BCI is that complex mouse movements such as click and drag, or scrolling, were not available. However, the researchers aim to improve on this in future work. The study was designed to not make use of any preloaded accessibility software; however, many tasks such as typing and scrolling can be made more efficient by using these features. Showing that this level of functionality is possible without accessibility software demonstrates the level of control that the study participants had.
Personal impact
Perhaps most importantly for the participants themselves, was their ability to use the BCI for tasks other than those in the original study design. One, who was a musician, was able to play music again through a keyboard piano app. Another participant managed to send their first text message through the tablet. This work has thus not only demonstrated technical control, but also the impact the technology had on the participants themselves.
The Cybathlon challenge
The lead author Paul Nuyujukian said: “It was great to see our participants make their way through the tasks we asked them to perform, but the most gratifying and fun part of the study was when they just did what they wanted to do– using the apps that they liked for shopping, watching videos or just chatting with friends.”
