'Minority Report' gestural computing pretty much here

By | December 9, 2011, 7:07pm PST

Summary: Forget the desktop environment with one or two screens. Oblong Industries’ spatial operating system extends your workspace to every available screen in a room. With the latest developments, the technology is a frontrunner for the future of the user interface.

You’ve seen it on the big screen and on TED, now the idea of manipulating digital data in 3-D space as if it were Play-Doh is taking a big leap into reality thanks to the visionary work of John Underkoffler, Chief Scientist, Oblong Industries.

The inventor of the futuristic computing interface used in the film Minority Report spoke yesterday at the Net:Work Conference in San Francisco.

Underkoffler argues that much of the sci-fi applies to the desktop of tomorrow, noting that the next big computing disruption should come in advances in user interfaces, because that’s “all you have.”

“Behind the scenes, computers and networks are still abstract machines that essentially flip switches, but people don’t think in the abstract.” We as humans tend to think in concrete terms, such as time and space. Therefore, it is Oblong’s duty, said Underkoffler, to bring computers to their rightful place — the real world.

To “de-abstract” the machine, we need to look at space and how humans use it when pointing their fingers to reference and connect to distant objects (or pixels). This most basic of gestures is the linchpin for Oblong’s spatial interface as it serves as the unifying means to access dozens of networked screens in a room.

The second idea ripe for scrutiny, said Underkoffler, is that despite the existence of computer networks, devices and displays are “solipsistic islands” isolated from one another when in fact they should be “socialized.”

“Part of our project will be to socialize the machine, but not in the sense of social networks. Let the computers talk to one another in a meaningful and rich way,” he said.

Oblong is refining its g-speak spatial operating environment, which has roots extending back to three decades of research at MIT Media Lab. The device-agnostic system is designed to allow for a spatial user interface that works with networked computers so that dozens of screens can be seamlessly utilized by multiple workers in proximity and remotely.

The system assigns every single pixel with a three-space set of coordinates that is unique from all other pixels in a room, providing for a way to relate different screens in response to gestural input.

This year, Oblong launched Mezzanine as a way to apply the technology to the work environment. Watch Underkoffler’s talk for the latest thinking on what the future of computing may look like:

Related:

frog creative chief: think outside the computer box
Three reasons why telepresence robots trump videoconferencing

Why the future of mobile is screenless, touchless

Kick off your day with ZDNet's daily e-mail newsletter. It's the freshest tech news and opinion, served hot. Get it.

Topics

Christopher Jablonski is a freelance technology writer.

Disclosure

Chris Jablonski

Christopher Jablonski has no business relationships, affiliations, investments, or other actual/potential conflicts of interest relating to the content posted so far on this blog.

Biography

Chris Jablonski

Christopher Jablonski is a freelance technology writer. Previously, he held research analyst positions in the IT industry and was the manager of marketing editorial at CBS Interactive. He's been contributing to ZDNet since 2003.

Christopher received a bachelor's degree in business administration from the University of Illinois at Urbana/Champaign. With over 12 years in IT, he's an expert on transformational technologies, particularly those influential in B2B.

11
Comments

Join the conversation!

Just In

when you impose a measurement paradigm on time and space,
wessonjoe 13th Dec
@myangeldust

it is no longer an abstract concept.
computers measure time in clock-cycles and space in bytes.
monitors measure time in frequency-cycles and space in pixels.
he is a scientist because he thinks he can teach the two to interact more harmoniously.
only a philosopher would view time and space in abstract terms.
we should learn to be harmonious also.

happy
.
0 Votes
+ -
Blah, blah, blah, Kinect gestures and the Internet, blah, blah, blah.

To de-abstract the machine we need to look at space and how humans use it when pointing their fingers to reference and connect to distant objects (or pixels). This most basic of gestures is the linchpin for Oblongs spatial interface as it serves as the unifying means to access dozens of networked screens in a room.

Part of our project will be to socialize the machine, but not in the sense of social networks. Let the computers talk to one another in a meaningful and rich way, he said.


Really? The only original thought here is to actually attach the gestures the same way that the horrid Johnny Mnemonic movie did long before Minority Report.

Oh wait, that's not actually original. The execution may be original. The design isn't.
@Letophoro

That's Hollywood make believe. This fellow is doing it for real in today's computer labs. Give the guy a break.
0 Votes
+ -
a long way to go
cymru999 10th Dec
groups of workers manipulating multiple screens with gestures - I don't say it is impossible since time proves nothing ever is BUT I think there is a long way to go - I work in an education environment and come across many people struggling with operating one screen! The second question is how desirable this is - our track record is of using technology to capitalise and make profits not to benefit mankind. The introduction of robotics could have meant people working fewer hours, enjoying more leisure time instead it led to redundancy and higher unemployment. Technological advances are great but can we ever learn to deploy them in socially responsible ways?
Might have at least mentioned the heliodisplay, or something similar, in tandem with mid-air gesture recognition. This, though, is rather fluffy.
Mr. Underkoffler's a funny guy. He thinks time and space are concrete terms. There's nothing more abstract than time/space. This guy is a chief scientist? Must be an abstract title within the Oblong hierarchy.
@myangeldust

it is no longer an abstract concept.
computers measure time in clock-cycles and space in bytes.
monitors measure time in frequency-cycles and space in pixels.
he is a scientist because he thinks he can teach the two to interact more harmoniously.
only a philosopher would view time and space in abstract terms.
we should learn to be harmonious also.

happy
.
They are likely to be passed by with some form of laptop webcam syncing app built into a future release of OS X or Windows. (That they will probably steal from a linux open source project ... //smile)
0 Votes
+ -
wiimote
mihovil@... Updated - 12th Dec
all those things are already possible using wiimote.... the great and cheap thingy just google to see what marvelous stuff can be done using it...
0 Votes
+ -
I can't help but see a room full of workers frantically gesticulating...

"I'm sorry, I do not understand that gesture. Please try it again with more feeling or issue a voice command"...

"I'm sorry, I do not understand that voice command. Please try it again with less emotion or make a gesture"...
say that in a family friendly forum?

Now that I have your attention...

Perhaps there will be a time when computers, through recognition of manual and face and body gestures, will be able to communicate with people without the use of the spoken or written language, and people who need to use sign language will be able to communicate with others who do not know sign language. So, "gestural computing" will be another form of input/output for computers, and would be quite beneficial to, not only the deaf and mutes, but to all people around the world who might want to learn "gestural communications", and then, have a gesture enabled computer translate the gestures to the spoken and written word. So, we could end up with a Siri-like device equipped with a Kinect-type device, which recognizes gestures and converts them into what we know as normal language, but, it could get translated into any language directly without having to go through an interpreter.

Of course, the gestural computing would have many other applications besides communications.
A few things to ponder.

1) Just try to hold your arms in the air for an eight hour day and see how long you really last.
2) Not everyone has two arms, two hands, or even 10 fingers.
3) Are you going to use this on the plane, train, or automobile? Can you imagine Starbucks with a bunch of people waving their arms around?

I think it is great to pursue this type of alternate input mode of communication. However, not all ideas are best in all situations.

Join the conversation!

Formatting +
BB Codes - Note: HTML is not supported in forums
  • [b] Bold [/b]
  • [i] Italic [/i]
  • [u] Underline [/u]
  • [s] Strikethrough [/s]
  • [q] "Quote" [/q]
  • [ol][*] 1. Ordered List [/ol]
  • [ul][*] · Unordered List [/ul]
  • [pre] Preformat [/pre]
  • [quote] "Blockquote" [/quote]
ie8 fix

The best of ZDNet, delivered

ZDNet Newsletters

Get the best of ZDNet delivered straight to your inbox

Facebook Activity

White Papers, Webcasts, & Resources
ie8 fix