Defining a notation system for multitouch input

Gemmell points to three problems when describing touch gestures to clients, within a development team or in documentation: drawings of gestures are difficult and take up a lot of room on a page; lots of words are required to describe even a simple gesture; and there's "ambiguity" when describing even common gestures, such as "swipe."

He has designed a complete system to describe touch commands, even covering the "split tap" gesture.

In touch-screen gestures, there’s also the (not so well-known) concept of a split tap; this is where you’re performing one action with one hand, and then (simultaneously) perform an action with the other hand, as a single gesture.

This is commonly used in VoiceOver, where for example the user can roll a finger over the keyboard until they hear that they’re arrived at the correct key, then tap a finger of their other hand anywhere on screen to enter that key. Split taps in VoiceOver allow for much easier text-entry where precision is only important for one hand.

I think the general concept of split input is interesting and useful, so I included it in the notation: I use a forward-slash to indicate the split. I don’t consider this to create ambiguity, since I only ever encode one gesture in a notation sequence, so I wouldn’t be using a forward-slash to mean “or”, as it can in English.

Of course, there's a detailed list of symbols for each gesture and examples of combinations. However, it will take me a while to quickly grok the Tap and Hold, then Drag, and then Tap elsewhere with another finger symbology.

Unlike the mostly touch interface of the iPad, the Macintosh concepts are confined for the moment to the MacBook/Pro models with trackpads that support multitouch. Apple's Human Interface Design Guidelines are sparse on the gesture front. Here are its suggestions to developers:

? Respond to gestures in a consistent way so that users know what to expect. In particular, avoid redefining the meaning of standard gestures in your application. ? Handle gestures as responsively as possible. Gestures should heighten the user’s sense of direct manipulation and provide immediate, live feedback. To achieve this, aim to perform relatively inexpensive operations for the duration of the gesture. ? Ensure that gestures apply to user interface elements of the appropriate granularity. Gestures are usually most effective when they target a specific object or view in a window, because such views are usually the focus of the user’s attention. Start by identifying the most specific object or view the user is likely to manipulate and make it the target of a gesture. Make a higher level or containing object a gesture target only if it makes sense in your application. ? Define custom gestures cautiously. A custom gesture can be difficult to discover and remember. If a custom gesture seems gratuitous, users are likely to avoid using it. If you feel you must define a custom gesture, be sure to make it easy to perform and easy to distinguish from standard gestures.

And in true Apple fashion, developers shouldn't rely totally on the availability of a specific gesture since users can turn off gestures in TrackPad Preferences.