Generative adversarial networks (GANs) technology, best known for making deepfake movies and photorealistic human faces, is being used by researchers at the USC Viterbi School of Engineering to improve brain-computer interfaces for persons with disabilities.
Deep fake (sometimes called deepfake) is an artificial intelligence technique for creating convincing pictures, audio, and video forgeries. The phrase is a combination of deep learning and fake, and it describes both the technique and the ensuing fraudulent content.
The scientists successfully taught an AI to generate synthetic brain activity data, according to a paper published in Nature Biomedical Engineering. Machine-learning algorithms can use the data, specifically neural signals termed spike trains, to improve the usability of brain-computer interfaces (BCI).
BCI systems analyze a person’s brain signals and translate them into commands, allowing the user to operate digital devices such as computer cursors using only their thoughts.
People with motor dysfunction or paralysis, as well as those suffering from locked-in syndrome, in which a person is completely cognizant but unable to move or communicate, may benefit from these devices.
BCI is presently accessible in a variety of forms, ranging from caps that measure brain impulses to devices implanted in brain tissue. New applications are being discovered all the time, ranging from neurorehabilitation to depression treatment.
Despite all of this promise, making these systems quick and reliable enough for real-world use has been difficult. BCIs, in particular, require a large volume of brain data as well as extensive training, calibration, and learning to make sense of their inputs.
When a company is ready to start commercializing a robotic skeleton, robotic arm or speech synthesis system, they should look at this method, because it might help them with accelerating the training and retraining. As for using GAN to improve brain-computer interfaces, I think this is only the beginning.
Laurent Itti
“Getting enough data for the algorithms that power BCIs can be difficult, expensive, or even impossible if paralyzed individuals are not able to produce sufficiently robust brain signals,” said Laurent Itti, a computer science professor and study co-author.
Another obstacle: the technology is user-specific and has to be trained from scratch for each person.
Generating synthetic neurological data
What if you could make synthetic neurological data, which is artificially computer-generated data that can “stand in” for real-world data?
Generative adversarial networks are a type of generative adversarial network. GANs, which are known for producing “deep fakes,” may generate an almost infinite amount of fresh, similar images through a trial-and-error process.
Shixian Wen, a Ph.D. student advised by Itti, wondered if GANs could also be used to generate training data for BCIs by generating synthetic neurological data that was indistinguishable from real data.
The researchers used one session of data recorded from a monkey reaching for an object to train a deep-learning spike synthesizer in an experiment reported in the publication. The synthesizer was then used to generate vast volumes of identical, albeit fictitious brain data.
To train a BCI, the researchers blended the synthesized data with modest quantities of additional real data from the same monkey on a different day or from a different animal. This procedure was far faster than the current standard approaches in getting the system up and operating.
The researchers discovered that using GAN-synthesized neural data increased the total training pace of a BCI by up to 20 times.
“Less than a minute’s worth of real data combined with the synthetic data works as well as 20 minutes of real data,” said Wen.
“It is the first time we’ve seen AI generate the recipe for thought or movement via the creation of synthetic spike trains. This research is a critical step towards making BCIs more suitable for real-world use.”
Furthermore, utilizing minimum additional brain data, the system quickly responded to new sessions, or subjects, following training on one experimental session.
“That’s the big innovation here creating fake spike trains that look just like they come from this person as they imagine doing different motions, then also using this data to assist with learning on the next person,” said Itti.
Beyond BCIs, GAN-generated synthetic data has the potential to speed up training and improve performance in other data-hungry fields of artificial intelligence.
“When a company is ready to start commercializing a robotic skeleton, robotic arm or speech synthesis system, they should look at this method, because it might help them with accelerating the training and retraining,” said Itti. “As for using GAN to improve brain-computer interfaces, I think this is only the beginning.”
The paper was co-authored by Tommaso Furlanello, a USC Ph.D. graduate; Allen Yin of Facebook; M.G. Perich of the University of Geneva and L.E. Miller of Northwestern University.
