Beyond the desktop: Ubuntu Linux's new Head-Up Display

Mark Shuttleworth, founder of Canonical Ubuntu Linux's parent company, has announced that Ubuntu will be adopting a radical new change to the interface that will do away with the "menu" in the Windows, Icons, Menus, Pointer (WIMP) interface, which has defined the desktop for the last thirty years.

Shuttleworth states, "The menu has been a central part of the GUI since Xerox PARC invented 'em in the 70?s. It's the M in WIMP and has been there, essentially unchanged, for 30 years. We can do much better!" This new interface, which will first appear as a beta in April's Ubuntu 12.04 release, is called Head-Up Display.

He explains:

Menus serve two purposes. They act as a standard way to invoke commands which are too infrequently used to warrant a dedicated piece of UI real-estate, like a toolbar button, and they serve as a map of the app's functionality, almost like a table of contents that one can scan to get a feel for 'what the app does'. Its command invocation that we think can be improved upon, and that's where we are focusing our design exploration.

As a means of invoking commands, menus have some advantages. They are always in the same place (top of the window or screen). They are organized in a way that's quite easy to describe over the phone, or in a text book ("click the Edit->Preferences menu"), they are pretty fast to read since they are generally arranged in tight vertical columns. They also have some disadvantages: when they get nested, navigating the tree can become fragile. They require you to read a lot when you probably already know what you want. They are more difficult to use from the keyboard than they should be, since they generally require you to remember something special (hotkeys) or use a very limited subset of the keyboard (arrow navigation). They force developers to make often arbitrary choices about the menu tree ("should Preferences be in Edit or in Tools or in Options?"), and then they force users to make equally arbitrary effort to memorize and navigate that tree.

No one would argue with that. So, what does Shuttleworth propose instead?

HUD will use a "vocabulary UI", or VUI. In this you'll start to type or say a command and in 12.04 LTS, the HUD starts a smart look-ahead search through the app and system (indicator) menus. This uses fuzzy matching, combined with a learning function so HUD will prioritize the actions you use do. This works with both the focused app, because that's where you probably want to act, and the global system functionality. So, for example, if, you're always saving your files, you could type Alt-F and instead of the file menu, you'd immediately get the file/save choice. Eventually you'll always be able to simply say "Save" and your word processor, spreadsheet, or what have you will save your current file.

You'll always be able to use HUD to get to other active programs, such as your instant message or Voice-over-Internet Protocol (VoIP) client through HUD, "without changing focus, because those apps all talk to the indicator system.' Shuttleworth claims that after "you've been using it for a little while it seems like it's reading your mind, in a good way."

Shuttleworth believes that voice will be an important part of HUD "Searching is fast and familiar, especially once we integrate voice recognition, gesture and touch. We want to make it easy to talk to any application, and for any application to respond to your voice. The full integration of voice into applications will take some time. We can start by mapping voice onto the existing menu structures of your apps. And it will only get better from there."

Still, Shuttleworth says "even without voice input, the HUD is faster than mousing through a menu, and easier to use than hotkeys since you just have to know what you want, not remember a specific key combination. We can search through everything we know about the menu, including descriptive help text, so pretty soon you will be able to find a menu entry using only vaguely related text (imagine finding an entry called Preferences when you search for "settings")."

It's not just speed though. Another goal, as it has been "in much of the Unity design has been to return screen real estate to the content with which the user is working; the HUD meets that goal by appearing only when invoked. Instead of cluttering up the interface ALL the time, let's clear out the chrome, and show users just what they want, when they want it."

The ideas for this, Shuttleworth continued, have been "inspired by many works of science, art and entertainment; from Minority Report to Modern Warfare and Jeff Raskin's Humane Interface. We hope others will join us and accelerate the shift from pointy-clicky interfaces to natural and efficient ones."

Although Shuttleworth doesn't say so, part of the reason why Ubuntu is making these changes is to make Ubuntu a more attractive option to smart TVs and smartphones and tablets. On these systems, the WIMP interface doesn't work that well. If HUD is successful, users will ultimately be able to use the same interface on any device from PC to smartphone with voice commands.

If that sounds more scary than neat, worry not. Shuttleworth isn't going to leave you in the lurch. "We'll resurrect the (boring) old ways of displaying the menu in 12.04, in the app and in the panel. In the past few releases of Ubuntu, we've actively diminished the visual presence of menus in anticipation of this landing. That proved controversial. In our defense, in user testing, every user finds the menu in the panel, every time, and it's obviously a cleaner presentation of the interface. But hiding the menu before we had the replacement was overly aggressive. If the HUD lands in 12.04 LTS, we hope you'll find yourself using the menu less and less, and be glad to have it hidden when you are not using it. You'll definitely have that option, alongside more traditional menu styles."

Related Stories:

TV for Human Beings: Ubuntu Linux

Can Ubuntu Linux win on smartphones and tablets?

Ubuntu Linux will try for the business desktop

Ubuntu Linux 11.10: Unity comes of age (Review)

New Desktop Interface Flops