Touch-a touch-a touch-a touch me
Richard O'Brien definitely didn't have the natural user interfaces in mind when he wrote the lyrics to Rocky Horror - or at least not the sort that as an industry we're currently feeling our way towards…
And feeling might be a more apt term than most. I've been struggling with NUI recently, feeling that something was missing. Perhaps I'd been looking at too many laptops and tablets that stopped at two and three finger gestures. Kinect was more promising, using the whole body and experimenting with more complex gesture languages, but it was for the living room, not for the enterprise, and we can't all stand 10 foot from a desk with a 40" monitor.
It took me a while to realise what was bothering me was that we're hardwired to work with both hands, able to make complex – and above all – independent motions with each hand.
Ttwo finger gestures like pinch-to-zoom are simplistic approaches that underestimate end users' dexterity. They're reductionist, producing a minimalist gesture language that doesn't even match the capabilities of the mouse. And yet, despite the limitations of gesture language to pinch and swipe, we're drawn to touch devices – and once you start touching one screen, it's easy to reach out to all the screens in your life (part of the reasoning behind the development of the new Windows 8 user interfaces).
Last week, though, things changed. I finally saw a company taking the idea of NUI and doing something innovative.
Adobe's upcoming suite of creative tablet tools does two things. Firstly it takes away the idea that tablets are limited to consumption (something I admit to being convinced of by the limitations of the iPad and the first Android tablets), and secondly it builds on work done with NUI research prototypes like MSR's Gustav to give us a true multi-hand gesture language.
With tools like the next release of Ideas and with the upcoming Photoshop Touch you'll use two hands, one to select and choose functions and manipulate them on the fly, the other to work with content. It's an approach that lets you change the width of a brush as you paint, and that lets you manipulate filters and settings while you move selections around a composition. It's a much more natural way of working, and one that's much closer to the way we work in the natural world.
That natural approach makes a lot of sense when working with slate-like tablets, which mimic natural objects. Need to draw at a different angle? Just rotate a tablet with one hand as you draw with the other. Multi-hand gestures mean you can work with the tablet at any angle, just like a piece of paper. They're just more natural. Demo devices running Photoshop Touch were on a stand at Adobe's MAX event, and the art produced by designers randomly doodling for five minutes on beta code was stunning.
Still, touch has one big flaw, even with Adobe's new tooling: the resolution of capacitive touch screens is too low for detailed work. That's where a future generation of tablets will help, tablet devices that mix capacitive touch and higher resolution pens using Wacom-style interfaces. Adobe demonstrated a hybrid touch/pen prototype at MAX 2011, which added a Wacom layer to a stock Galaxy Tab 10.1. It wasn't perfect – for one thing Android doesn’t have blunt-touch blocking – but it did give an idea of what combining the two techniques could offer.
We get stuck in thinking of NUI as one technology as a time. It turns out that hybrid NUI tooling is much more natural, much easier to grasp and learn. Working with pen and paper is something we understand from a very early age. Surely it should be the same for pen and screen?
Simon Bisson