Mind-reading AI turns thoughts into words using a brain implant
Mind-reading AI turns thoughts into words using a brain implant
By
Joseph
Makin at the University of California, San Francisco, and his colleagues used
deep learning algorithms to study the brain signals of four women as they
spoke. The women, who all have epilepsy, already had electrodes attached
to their brains to monitor seizures.
Each woman
was asked to read aloud from a set of sentences as the team measured brain
activity. The largest group of sentences contained 250 unique words.
The team
fed this brain activity to a neural network algorithm, training it to identify
regularly occurring patterns that could be linked to repeated aspects
of speech, such as vowels or consonants. These patterns were then fed
to a second neural network, which tried to turn them into words to form a
sentence.
Each woman
repeated the sentences at least twice, and the final repetition didn’t
form part of the training data, allowing the researchers to test
the system.
Each time a
person speaks the same sentence, the brain activity associated will be similar
but not identical. “Memorising the brain activity of these sentences wouldn’t
help, so the network instead has to learn what’s similar about them so that it
can generalise to this final example,” says Makin. Across the four women, the
AI’s best performance was an average translation error rate of 3 per cent.
Makin says that using a small number of
sentences made it easier for the AI to learn which words tend to follow others.
For example,
the AI was able to decode that the word “Turner” was always likely to
follow the word “Tina” in this set of sentences, from brain activity alone.
The team
tried decoding the brain signal data into individual words at a time, rather
than whole sentences, but this increased the error rate to 38 per cent even for
the best performance. “So the network clearly is learning facts about which
words go together, and not just which neural activity maps to which words,”
says Makin.
This will
make it hard to scale up the system to a larger vocabulary because each
new word increases the number of possible sentences, reducing accuracy.
Makin says
250 words could still be useful for people who can’t talk. “We want to deploy
this in a patient with an actual speech disability,” he says, although it is
possible their brain activity may be different from that of the women in this
study, making this more difficult.
Sophie
Scott at University College London says we are a long way from being able to
translate brain signal data comprehensively. “You probably know around 350,000
words, so it’s still an incredibly restricted set of speech that they’re
using,” she says.
Journal
reference: Nature Neuroscience,
DOI: 10.1038/s41593-020-0608-8
Read more: https://www.newscientist.com/article/2238946-mind-reading-ai-turns-thoughts-into-words-using-a-brain-implant/#ixzz6IJgIrydV
Read more: https://www.newscientist.com/article/2238946-mind-reading-ai-turns-thoughts-into-words-using-a-brain-implant/#ixzz6IJgIrydV
Comments
Post a Comment