Facebook's 'mind-reading' tech startup deal could completely change how we control computers
Earlier this week, Facebook announced it was acquiring CTRL-labs. The deal was variously reported as either the social media giant acquiring a "mind-reading tech company" or an outfit that makes "neural wristbands". In fact, the reality is much more complex -- and could potentially signal the start of a complete overhaul of how we interact with technology.
Social Media
The size of the acquisition hasn't been disclosed, but CNBC reported it was worth between $500m and $1bn, citing sources familiar with the deal. It may seem like a lot of money for a company whose developer kit is less than a year old, but Facebook isn't shy of taking expensive punts on what it believes are the frontrunners in emerging technologies -- take its $2bn acquisition of VR gaming company Oculus Rift, for example.
SEE: How to implement AI and machine learning (ZDNet special report) | Download the report as a PDF (TechRepublic)
So what's the CTRL-labs' appeal for Facebook?
CTRL-labs is one of the growing number of companies in a space known variously as brain computer interfaces (BCIs) or brain machine interfaces (BMIs).
"We spend a lot of time trying to get our technology to do what we want, rather than enjoying the people around us. We know there are more natural, intuitive ways to interact with devices and technology. And we want to build them. It's why we've agreed to acquire CTRL-labs. They will be joining our Facebook Reality Labs team where we hope to build this kind of technology, at scale, and get it into consumer products faster," Andrew Bosworth, Facebook's VP of virtual and augmented reality, wrote in a blog post.
"The vision for this work is a wristband that lets people control their devices as a natural extension of movement," he said.
As every generation of technology has been subject to a shift in input mechanisms -- from the keyboard to the mouse to touchscreen to voice -- CTRL-labs hopes it's identified the next major way we'll be interacting with our technology: hands-free gestures.
Typically, BCI systems use either chips implanted in the brain, or electrode-carrying caps worn on the scalp, to collect signals from the brain and translate them into emotional states, communications, or the user's intended movements. CTRL-labs is focusing on the latter, but it's doing it in a different way to many of its competitors. While non-invasive BCIs (those that don't involve a chip directly implanted in the brain) typically use EEG to gather data, CTRL-labs uses EMG. EEG (electroencephalography) measures electrical signals in the brain, while EMG (electromyography) measures those same impulses as they travel down the nerves in the body.
And, rather than use head-worn gear to pick up neural signals, CTRL-labs uses devices worn on the wrist to pick up the neural signals as they arrive at the muscles in the users' hands.
"If you think of your arms and hands as the pipe of information from your brain to the world, the primary way we as humans affect control in the world and make big changes is by moving our muscles, and in particular, our hands. The technology is a way of capturing that information, decoding it, understanding it, and then short-circuiting the path that we currently use to interact with technology," Adam Berenzweig, head of R&D at CTRL-labs, told ZDNet earlier this month.
CTRL-labs' bracelets will gather signals that control hand movements, and pass them onto whichever computer system is connected. "We're cutting out that middleman and decoding neural signals to do whatever you want to do with machines," Berenzweig added.
AR + VR
How might that work in the real world? People using VR could control their systems using their natural movements, rather than carry separate handheld controllers, or people using headsets like Google Glass could navigate with a simple flick of a finger. Gamers and office workers alike could do away with hardware like joypads and mice in favour of making a few gestures -- like a touchscreen, but with no need for touch of any kind. In essence, every piece of technology becomes hands-free. For those imagining a Minority Report style future of exaggerated hand-waving, fear not -- the system can pick up "tiny wiggles" as much as larger gestures.
"Each of those stages [of technology development] has a new human computer interface paradigm shift… We believe this input technology we have is a perfect fit to be the input for wearable computing, where your hands are the best controllers. You have your hands on all the time, and you don't need another device -- you have your own abilities with high precision, high dexterity, expressive hand movements," Berenzweig said.
"The modern VR systems are just great, they're visually beautiful and immersive, you put on a headset and you're feel like you're there and then you still have these sticks where your hands should be. The input side is lagging behind the visual side," Berenzweig added.
While CTRL-labs' likely interest for Facebook is VR -- the startup will join Facebook's VR unit -- the social network's interest in brain machine interface extends well beyond its latest purchase. In 2017, it was reported the company had a team of 60 people working on brain computer interfaces to allow people to type at 100 words a minute just using brain signals.
In July of this year, the company posted an update on the project, saying it had a functioning system, but one which recorded only a small set of words and phrases. While it's a separate strand of BCI research to CTRL-labs' own, the two are both subject to a common driver: the ability to help those with nerve damage to communicate again. In the case of Facebook's thought-typing program, researchers had previously worked on helping those with locked-in syndrome to communicate and were researching how to help people with neural conditions speak again using BCIs; for CTRL-labs, the device could potentially be used to help those with damage to their nerve pathways overcome paralysis.
"I think this technology is going to have a big impact on people's lives as an assistive device across a pretty wide range of pathologies and ways in which people struggle with using their hands or motor impairment. It's not our focus as a company -- we're focused on a mass consumer market, and think about the technology as something that billions of people would be using on a daily basis -- but we're aware of all these quite enticing and impactful clinical applications," Berenzweig said.
As long as there are some intact connections from the brain to the muscles of the hands, the CTRL-Labs system should be able to work. "We have partnerships with clinical research labs and we have a board of advisors with a number of research neuroscientists who are beginning to use this in their labs, and this hopefully this will lead into other products and clinical work," he said.
"As far as what kinds of things this could be used to address, [it could include] stuff like muscular dystrophy, MS, ALS, Parkinson's -- anything where there's an impairment but the nerves are intact and functioning to some degree," said Berenzweig, adding that the system could also potentially be used in stroke rehabilitation.