'Minority Report' gestural computing pretty much here

'Minority Report' gestural computing pretty much here

Summary: Forget the desktop environment with one or two screens. Oblong Industries' spatial operating system extends your workspace to every available screen in a room. With the latest developments, the technology is a frontrunner for the future of the user interface.

SHARE:

You've seen it on the big screen and on TED, now the idea of manipulating digital data in 3-D space as if it were Play-Doh is taking a big leap into reality thanks to the visionary work of John Underkoffler, Chief Scientist, Oblong Industries.

The inventor of the futuristic computing interface used in the film Minority Report spoke yesterday at the Net:Work Conference in San Francisco.

Underkoffler argues that much of the sci-fi applies to the desktop of tomorrow, noting that the next big computing disruption should come in advances in user interfaces, because that's "all you have."

"Behind the scenes, computers and networks are still abstract machines that essentially flip switches, but people don’t think in the abstract." We as humans tend to think in concrete terms, such as time and space. Therefore, it is Oblong's duty, said Underkoffler, to bring computers to their rightful place -- the real world.

To "de-abstract" the machine, we need to look at space and how humans use it when pointing their fingers to reference and connect to distant objects (or pixels). This most basic of gestures is the linchpin for Oblong's spatial interface as it serves as the unifying means to access dozens of networked screens in a room.

The second idea ripe for scrutiny, said Underkoffler, is that despite the existence of computer networks, devices and displays are "solipsistic islands" isolated from one another when in fact they should be "socialized."

"Part of our project will be to socialize the machine, but not in the sense of social networks. Let the computers talk to one another in a meaningful and rich way," he said.

Oblong is refining its g-speak spatial operating environment, which has roots extending back to three decades of research at MIT Media Lab. The device-agnostic system is designed to allow for a spatial user interface that works with networked computers so that dozens of screens can be seamlessly utilized by multiple workers in proximity and remotely.

The system assigns every single pixel with a three-space set of coordinates that is unique from all other pixels in a room, providing for a way to relate different screens in response to gestural input.

This year, Oblong launched Mezzanine as a way to apply the technology to the work environment. Watch Underkoffler's talk for the latest thinking on what the future of computing may look like:

Related:

frog creative chief: think outside the computer box Three reasons why telepresence robots trump videoconferencing Why the future of mobile is screenless, touchless

Topics: Operating Systems, Hardware, Software

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

11 comments
Log in or register to join the discussion
  • RE: 'Minority Report' gestural computing pretty much here

    Blah, blah, blah, Kinect gestures and the Internet, blah, blah, blah.<br><br> [i]To de-abstract the machine we need to look at space and how humans use it when pointing their fingers to reference and connect to distant objects (or pixels). This most basic of gestures is the linchpin for Oblongs spatial interface as it serves as the unifying means to access dozens of networked screens in a room. <br><br> Part of our project will be to socialize the machine, but not in the sense of social networks. Let the computers talk to one another in a meaningful and rich way, he said.[/i]<br><br>Really? The only original thought here is to actually attach the gestures the same way that the horrid Johnny Mnemonic movie did long before Minority Report.<br><br>Oh wait, that's not actually original. The execution [b]may[/b] be original. The design isn't.
    Letophoro
    • RE: 'Minority Report' gestural computing pretty much here

      @Letophoro

      That's Hollywood make believe. This fellow is doing it for real in today's computer labs. Give the guy a break.
      kenosha77a
  • a long way to go

    groups of workers manipulating multiple screens with gestures - I don't say it is impossible since time proves nothing ever is BUT I think there is a long way to go - I work in an education environment and come across many people struggling with operating one screen! The second question is how desirable this is - our track record is of using technology to capitalise and make profits not to benefit mankind. The introduction of robotics could have meant people working fewer hours, enjoying more leisure time instead it led to redundancy and higher unemployment. Technological advances are great but can we ever learn to deploy them in socially responsible ways?
    cymru999
  • RE: 'Minority Report' gestural computing pretty much here

    Might have at least mentioned the heliodisplay, or something similar, in tandem with mid-air gesture recognition. This, though, is rather fluffy.
    QuadFather
  • RE: 'Minority Report' gestural computing pretty much here

    Mr. Underkoffler's a funny guy. He thinks time and space are concrete terms. There's nothing more abstract than time/space. This guy is a chief scientist? Must be an abstract title within the Oblong hierarchy.
    myangeldust
    • when you impose a measurement paradigm on time and space,

      @myangeldust

      it is no longer an abstract concept.
      computers measure time in clock-cycles and space in bytes.
      monitors measure time in frequency-cycles and space in pixels.
      he is a scientist because he thinks he can teach the two to interact more harmoniously.
      only a philosopher would view time and space in abstract terms.
      we should learn to be harmonious also.

      :)
      .
      wessonjoe
  • RE: 'Minority Report' gestural computing pretty much here

    They are likely to be passed by with some form of laptop webcam syncing app built into a future release of OS X or Windows. (That they will probably steal from a linux open source project ... //smile)
    awilson77584
  • wiimote

    all those things are already possible using wiimote.... the great and cheap thingy <img border="0" src="http://www.cnet.com/i/mb/emoticons/happy.gif" alt="happy"> just google to see what marvelous stuff can be done using it... <img border="0" src="http://www.cnet.com/i/mb/emoticons/happy.gif" alt="happy">
    mihovil1
  • RE: Gestural computing . . .

    I can't help but see a room full of workers frantically gesticulating...

    "I'm sorry, I do not understand that gesture. Please try it again with more feeling or issue a voice command"...

    "I'm sorry, I do not understand that voice command. Please try it again with less emotion or make a gesture"...
    gitwut
  • "a room full of workers frantically gesticulating..."? Are you allowed to

    say that in a family friendly forum?

    Now that I have your attention...

    Perhaps there will be a time when computers, through recognition of manual and face and body gestures, will be able to communicate with people without the use of the spoken or written language, and people who need to use sign language will be able to communicate with others who do not know sign language. So, "gestural computing" will be another form of input/output for computers, and would be quite beneficial to, not only the deaf and mutes, but to all people around the world who might want to learn "gestural communications", and then, have a gesture enabled computer translate the gestures to the spoken and written word. So, we could end up with a Siri-like device equipped with a Kinect-type device, which recognizes gestures and converts them into what we know as normal language, but, it could get translated into any language directly without having to go through an interpreter.

    Of course, the gestural computing would have many other applications besides communications.
    adornoe
  • RE: 'Minority Report' gestural computing pretty much here

    A few things to ponder.

    1) Just try to hold your arms in the air for an eight hour day and see how long you really last.
    2) Not everyone has two arms, two hands, or even 10 fingers.
    3) Are you going to use this on the plane, train, or automobile? Can you imagine Starbucks with a bunch of people waving their arms around?

    I think it is great to pursue this type of alternate input mode of communication. However, not all ideas are best in all situations.
    ChodaBoy