AI can now read the thoughts of paralysed patients as they imagine they are writing

A team of neuroscientists has demonstrated that an algorithm could be trained to recognise the mental preparation for the act of writing letters, and that it could be twice as effective as existing technologies.
Written by Daphne Leprince-Ringuet, Contributor

Handwriting is becoming a rare skill in the digital age. But researchers have now discovered a new application that could significantly improve the way tetraplegic people, who are often also unable to speak, communicate with the outside world.

At the Society for Neuroscience's annual meeting in Chicago this week, a team of neurologists presented a new tool that could read out the sentences formed by a volunteer paralyzed from the neck down, in double the average speed recorded for existing technologies.

The key to success? The volunteer's imagination: he was asked to imagine that he was moving his arm to hand-write each letter of the alphabet, one at a time, with an imaginary pencil.

SEE: How to implement AI and machine learning (ZDNet special report) | Download the report as a PDF (TechRepublic)

Writing, since it's a movement, requires a certain cerebral organization that has already been located in previous studies as happening in the primary motor cortex. This 'preparatory state' to the act of writing is what the researchers used for their new tool.

Because letters have distinct trajectories, the attempt to write -- even if only a mental one -- triggers types of brain activity that are different enough to be separately recorded via microelectrodes and used to train a neural network.

The network was then incorporated into a brain-computer interface (BCI) so that the brain's attempts to write could be recognized and translated into text by a machine in real time. 

The computer read the volunteer's sentences with 92% accuracy at a speed of 66 characters per minute, according to the neurologists.

Mistakes were mostly confined to letters that look similar, such as 'g' and 'q', they said, because the preparatory states for those are identical, suggesting that those letters require extra effort in the physical act of writing to be completed.

Still, the handwriting tool is almost twice as fast as regular technologies, which currently manage to decrypt paralyzed patients' attempts to communicate at 39 characters per minute -- three times slower than natural handwriting for a healthy adult.

"These preliminary results suggest that a handwriting BCI could be accurate enough to achieve high communication rates," the researchers said.

Patients can already use BCIs -- albeit much slower ones -- to communicate, but they aren't required to imagine that they are writing by hand. 

Instead, they are presented with letters on a screen, and can select them by mentally pointing and clicking a cursor on the screen. To do so, electrodes record and translate the part of the brain responsible for movement. 

SEE: Mind-reading systems: Seven ways brain computer interfaces are already changing the world

The researchers expect the new tool to refine its translations and achieve even higher speeds with practice. 

However, there are long-standing challenges that BCI technology will have to tackle before it can be adopted widely -- notably, how cumbersome it is. It effectively requires an in-house team of neurophysiologists and engineers, and cannot be performed outside of specialised laboratories.

So although the new handwriting tool is an exciting development, there's a long way to go before it becomes a routine communication facilitator for tetraplegic patients.

Editorial standards