Emotiv's man-machine melding interface technology
The brain signal data acquired via the headset (prototype at right) is streamed wirelessly to a receiver box connected to a computer. Proprietary software and silicon is used to measure and interpret the electrical signals, which represent signatures for various states or activities, said Randy Breen, chief product officer. Game developers can calibrate the sensitivity of settings and determine interactions based on the Emotiv data.
The Emotiv team was not forthcoming about the underlying technology other than to say that it is the "interface between neurobiology and statistics."
The first toolkit, dubbed Expressiv (note: the last of an "e" is intended...like Flickr), identifies facial expressions, such as blink, smile, wink, horizontal eye movement, clenching and eyebrow movement. The data could be used to animate avatars in massively multiplayer online (MMO) games that exhibit a person's behavior, allowing for more realistic virtual chatting or machine interaction with some form of artificial intelligence, Breen said.
Emotiv Systems team members in San Francisco: Nam Do, Randy Breen, David Allsop, Tan Le and Marco Della Torre. Emotiv is the brainchild of neuroscientist Allan Snyder, chip designer Neil Weste and technology entrepreneurs Tan and Nam. Snyder was awarded the prestigious 2001 Marconi International Prize for describing the way light travels along optical fibers, which led to the modern telecommunications revolution. Weste was formerly head of the Bell Labs VLSI Department and Tan and Nam co-founded SASme, which provided SMPP platforms to telecommunication carriers and content aggregators in Asia. Product chief Breen was formerly head of development at LucasArts, and vice president of engineering Steve Sapiro, was a founder of CAE Systems Inc., proCASE Corp., SiidTech Inc. and Stelar Tools Inc.
The second toolkit, Affectiv, goes beyond recognizing facial gestures and detects a player's emotional states. For example, if numerous people at a certain point in a game are frustrated, as indicated by the brain signatures detected, the game could automatically adapt by easing up the difficulty. A game that employed humor could use the data to determine if players were having the expected experience and adjust game play to optimize the game play. If excitement is detected, the game animation could be cranked up to enhance the effect, or a role playing game the character could reflect the emotional state of the player.
The technology could also be applied to audience research, such as determining emotive reactions to a film or commercial, or to unconsciously rating music on iTunes, Breen said. For healthcare, the technology could assist in determining the mental state of unconscious patients.
The third toolkit, Cognitiv, allows players to manipulate virtual objects using mind control. "The noninvasive technology can distinguish different thoughts," Tan said, "but not just on base states like an EEG. You can think specific thoughts of moving or pulling and it distinguished between them."
"We can detect for a conscious idea, and project that into the image," Breen added. Cognitiv supports 12 actions, such as lift, drop, let, rotate clockwise, rotate counterclockwise, push and pull, that can be programmed into objects. "It's not the same precision as a mouse or joystick, but you have the ability to think of something and make it happen," said David Allsop, the Emotiv research engineer who demoed the Emotiv technology. Multiple actions can be combined, such as lifting and rotating.
I wasn't permitted to try to move objects myself, but a demo of the technology integrated into a commercial Harry Potter game didn't appear to be smoke and mirrors. This part of the development requires some amount of calibration and explicit man-machine training for game players. "Fantasy games come with a lot of mind control," Breen said. "Humans evolve into creatures with powers beyond bodies, and we extend those ideas in literal way. In a MMO, for example, 50 people in concert could raise a temple." The mind control and other input devices can be used in tandem, such as navigating around a space with a joystick and picking up object via "thinking."
It turns out that brain signatures for specific actions are relatively generic. Marketplaces for trading and selling Cognitiv action signatures could emerge, although that would not help players who are too unfocused in their thinking, lacking in powers of concentration, to move anything. If Emotiv is successful with its technology, players might complete for who can move objects the fastest and with the least body movement.
Currently the Emotive Developer Kit is in alpha phase with a few developers. The headset and other electronics will be downsized and included in Emotiv-enabled games in 2008. Nam declined to talk about pricing, other than to say it would be in the "consumer range."
As a first generation human-machine interface integrated with games, Emotiv, which has been working three years on its technology, will take a few years to shake out the cobwebs, but the face of gaming and other disciplines will be forever changed. I'm still waiting for the kind of mind control that allows me to move my real furniture without lifting finger.