Speaking again after paralysis, thanks to a brain implant and AI

Why is speech so hard to restore?

Severe paralysis can take away a person's arms and legs, but also their voice. The brain keeps preparing the words, yet the message no longer reaches the muscles of the mouth and throat. "Speech neuroprostheses" try to read that message directly from the brain and turn it into words.

Until now, these devices struggled to combine natural speed with expressiveness. Spelling out words letter by letter is slow and tiring. Natural conversation needs speed, a voice and a face that moves, not only a string of letters on a screen.

What did the researchers do?

The team worked with a person in a clinical trial who has severe paralysis of the limbs and voice. A very dense grid of electrodes, placed on the surface of the brain area that controls speech, records the activity of that area.

The person tries to say sentences silently. Deep-learning models (AI that learns from examples) are trained to link those brain signals to what they are trying to say. The system produces three outputs in real time: text, synthetic speech and the movements of a virtual face, an avatar.

For the voice, the team personalised it so that it sounds like the person's voice before their injury.

What do the results show?

For text, decoding reaches a median rate of 78 words per minute with a large vocabulary, and a median word error rate of 25%. In other words, in the typical case about one word in four is wrong, so the text is not yet perfect.

The synthesised speech is described as intelligible and fast. The avatar reproduces mouth and face movements, both for speaking and for communicative gestures that are not speech.

The models reached this performance with less than two weeks of training. That is short for a system that has to learn the link between brain signals and a large vocabulary.

What does this change for patients?

The aim is to restore fuller, "embodied" communication, with voice, face and the rhythm of a real conversation. For people locked inside their bodies, this could be a huge gain in independence, if the approach is confirmed.

Today it is a research demonstration. It is not a product available to patients.

What are the limits?

The study involves one person. We do not know whether results will be the same in others, with different brains and different causes of paralysis. The device requires brain surgery and heavy equipment.

A 25% error rate also means the text still needs checking. Larger, longer trials outside the laboratory are still needed to show how well it works.