Apple Patent app reveals iPhone-compatible Multi-Touch Gesture Dictionary

If you've ever taken guitar lessons, you know that various musical chords are formed via various fingering positions.

The "gestural language" Apple uses for onscreen iPhone navigation offers a potential for a simular type of gestural language for working with various device functions.

These concepts are described in a newly published Apple Patent application entitled Multi-Touch Gesture Dictionary.

Parsing the Abstract gives us a taste of what is being described:

A multi-touch gesture dictionary is disclosed herein. The gesture dictionary can include a plurality of entries, each corresponding to a particular chord. The dictionary entries can include a variety of motions associated with the chord and the meanings of gestures formed from the chord and the motions.

The gesture dictionary may take the form of a dedicated computer application that may be used to look up the meaning of gestures. The gesture dictionary may also take the form of a computer application that may be easily accessed from other applications. The gesture dictionary may also be used to assign user-selected meanings to gestures.

Also disclosed herein are computer systems incorporating multi-touch gesture dictionaries. The computer systems can include, desktop computers, tablet computers, notebook computers, handheld computers, personal digital assistants, media players, mobile telephones, and the like.

More explanation follows:

Multi-touch gestures may be considered to include at least two phases that, taken together in sequence, signal the beginning and completion of a particular gesture. The first phase of a multi-touch gesture can include presenting a specific combination of hand parts, i.e., fingers, thumbs, etc. in a particular configuration.

In some embodiments, this may include placing the hand parts down on the multi-touch surface. The second phase of the gesture can include, for example, motion of the specific hand parts. This motion may take the form of lateral motions such as rotation, translation, scaling (expansion and contraction), etc. Again, in some embodiments, this may comprise moving the hand parts around on the multi-touch surface. In such embodiments, the second phase of the gesture may also comprise vertical motions (relative to the multi-touch surface) such as tapping, double-tapping, etc.

For convenience, the first phase, e.g., the starting position, number, and configuration of all the hand parts used for a particular gesture, will be referred to herein as a chord. Also for convenience, the hand parts will be referred to as fingers, although this also includes thumbs, palm heels, etc.

Therefore, in the examples described herein, a chord can include a set of fingers from either or both hands that initially contact a multi-touch surface prior to motion on the multi-touch surface. In many multi-touch systems the chord may uniquely specify a set of gestures that belong to the combination of fingers and orientations making up the chord.

Each of a user's hands can execute twenty-five or more chords. For example, five fingers that can be independently raised or lowered give rise to twenty-five combinations. Additional chords may be distinguished by whether only the fingertips are in contact with the surface or whether the length of the finger is flattened against the surface.

Further chords may be distinguished based on whether the fingertips are placed on the surface close together or spread apart. Still other distinctions may be possible. For example, modifier keys (e.g., the Ctrl, Alt, Shift, and Cmd keys of a keyboard) may be used to distinguish different chords. The modifier keys may include keys on a conventional keyboard or may include buttons or touch or force sensitive areas or other toggles located on the device. However, some of these chords may be more difficult to execute than others, and various identification and classification problems can arise for the device, particularly in the case of closed versus spread fingertips.

Many chords can have at least thirteen different motions associated with them. For example, a two-finger chord (for example, the index and middle fingers) could have specific meaning or action assigned to the lateral motions that include rotation, translation, and scaling.

Rotation (clockwise and counter-clockwise) of the two-finger chord gives rise to two unique meanings or actions. Translation (left, right, up, down, and four diagonals) gives rise to at least eight unique meanings or actions. Scaling (contraction or expansion) also gives rise to two meanings or actions. The vertical motion of a chord may comprise lifting the fingers of the chord off the multi-touch surface almost immediately after they had touched down, (e.g., tapping the multi-touch surface with the chord) or multiple taps, etc.

With each hand able to execute twenty-five or more chords, and with each chord having thirteen or more motions associated therewith, there may be over three hundred possible gestures for each hand. Many more gestures are possible if both hands are used together. This gives rise to the gesture language referenced above.

One approach to creating a gesture dictionary indexes the dictionary using the chords, much as a textual dictionary uses the alphabet. For example, just as there may be a particular number of words that start with a particular letter, so there may be a particular number of gestures that start with a particular chord.

These gestures may be presented to a user in a way that facilitates rapid assimilation by the user. For example, template 100 for a combination graphical and textual dictionary entry for a given chord is illustrated in FIG. 1.

Guess I need to show you Figure 1. Hey I get paid to do this stuff, plus it's cool. So no prob:

Template 100 can include an indication 114 of a given chord and a plurality of indications 101-113 corresponding to motions associated with the given chord, which may be called motion icons. In this example, the motions include translation upward and to the left 101, translation upward 102, translation upward and to the right 103, translation to the left 104, tapping 105, translation to the right 106, translation downward and to the left 107, translation downward 108, translation downward to the right 109, counter-clockwise rotation 110, clockwise rotation 111, expansion 112, and contraction 113. Other motions can also be included in template 100.

Alternatively, motions that may not apply to a given chord or that may be difficult to execute with a given chord can be omitted. The arrangement of the motion icons may be organized in a logical and consistent manner for all of the dictionary entries so as to provide the user with a basically constant layout, which allows the user to always know where to look to get the meaning of a gesture.

Each of FIGS. 2-5 shows an exemplary dictionary entry for four different chords. In each of these exemplary dictionary entries, the textual descriptions of the motions from the template of FIG. 1 are replaced with the "meaning" of a particular gesture. The meanings may take the form of commands, strings of commands, or other activities such as entry of particular text, etc.

Now I will post Figures 2-5. That should give you an idea of what is going on here:

And finally, Figure 10 will show you sketches of some devices Apple would appear to envision as suited for these "gestural dictionaries":