Here's how Jakob Nielsen's Windows 8 findings should inform developers

I must admit to have grown a little bored with writing about Windows 8 and Windows RT as a product. It's true to say that either of them are complicated, and at the end of it it doesn't matter whether you read my old opinion of Windows 8, my new opinion, or someone else's opinion. They're always subjective and the market on aggregate gets to decide.

But Jakob Neilsen is different -- or rather, what he has to say about Windows 8 is different because his method isn't subjective. It's based on research. More to the point, he's rather good at researching usability. I've admired his work for years, and I'm not alone -- he's probably the preeminent expert on computer usability and his opinion matters.

And you can now find a new report on this website:"Windows 8 -- Disappointing Usability for Both Novice and Power Users". Spoiler: the upshot is that usability on Windows 8 is quite grim.

Someone else who's opinion does matter is my ZDNet colleague Adi Kingsley-Hughes. He's posted a a great summary of Jakob's report.

But somewhere in Jakob's report are some gems that we can take out and apply to our Windows Store apps to enhance usability within the constraints of what we get given.

"Double desktop"

The general shape of Jakob's points in this area what most developers can infer for themselves. The duality of Old Windows and New Windows (*cough* Metro *cough*) causes a problem when getting into Windows 8. Everyone talks about the duality problem. As I've mentioned before, my own view on this is that Microsoft's engineers' hands were tied -- what else could they do in order to try and manhandle Windows from the old PC world into the new post-PC world.

I won't dwell no this issue. This is just the world we have to work in. The other issues he raises we can (mostly) do something about.

"Lack of multiple windows"

This has got a quite funny part in it:

Also, the main UI restricts users to a single window, so the product ought to be renamed "Microsoft Window."

Well, I laughed. Anyway, he's right. Joking aside, here's the more serious part:

When users can't view several windows simultaneously, they must keep information from one window in short-term memory while they activate another window. This is problematic for two reasons. First, human short-term memory is notoriously weak, and second, the very task of having to manipulate a window -- instead of simply glancing at one that's already open -- further taxes the user's cognitive resources.

This is really one of my main problems with the way apps are built for Windows 8. Everything is hidden away. And hiding something, as Jakob says, means you a) have to remember where it is and b) you have a context switching problem. Going to find information as a user is an intensely complex task. You have to stash the state of what you were doing, recall the commands associated with finding the new information, execute those commands, process what you find, recall the commands to flip back, and execute those commands to flip back. That assumes that what you were doing worked. You can mitigate this in Windows 8 by designing your UI so that information is visible and in-hand.

In this sense, Windows 8 isn't any different to iOS on the iPad. You still have single-serving apps, you still have just one window, and you can still make users go through complex processes to find what they need. In short, you can fix most of this problem just by being good at UX.

(Jakob's main point here is that you can't do things like use multiple monitors to show an Excel workbook, a Word document and a web browser side-by-side. You can fix the intra-app problem, you just can't fix the single serving app problem.)

"Flat Style Reduces Discoverability"

This was one I hadn't considered before. I don't particularly like the way Metro looks, but it does reduce a the amount of effort developers have to do. Making stuff look good in the Metro aesthetic is very easy.

Of course, in Metro, everything is flat. Gone is the faux 3D trickery introduced in Windows 95. Whilst Jakob liked part of it -- "I do think Metro/Modern has more elegant typography than past UI styles and that the brightly colored tiles feel fresh" -- his problem here was that users didn't know that clickable stuff was clickable.

Specifically, here's the Settings flyout in Windows 8:

He then said:

Where can you click? Everything looks flat, and in fact "Change PC settings" looks more like the label for the icon group than a clickable command. As a result, many users in our testing didn't click this command when they were trying to access one of the features it hides.

What this highlights to me is that the Settings flyout is actually wrong. Buttons in Windows 8 are either circular or square. They may be buried in the app bar from time-to-time, but actually what is a target and isn't a target is generally obvious. Those six Setting flyout icons should have circles round them. "Change PC Settings" should be a button. So I'm not going with Jakob on this one -- Microsoft needs to fix it.

"Low Information Density"

If there's one crime that means we're allowed to kill Metro with fire, put the ashes in a box, put the box in the ocean, decide that's not enough, take the box out of the ocean, wrap it in lead, put it in a space rocket and fire it at the heart of the sun, it's the crime of "low information density". I wrote about this back in May.

What is meant by "information density" is given a set amount of real estate, how tightly packed is the information available to the user. Here's what Jakob had to say:

Despite running on a huge 10.6-inch tablet, Bing Finance shows only a single story (plus 3 stock market quotes) on the initial screen. The Los Angeles Times is not much better: this newspaper app's initial screen is limited to 3 headlines and an advertisement. In fact, they don't even show the lead story's full headline and the summary has room for only 7 words. Come on, this tiny amount of news is all you can fit into 1366 × 768 pixels?

Jakob also mentions that he thinks that developers are encouraged to drive towards low-density display. I don't think this is the problem per se. The problem here is that Metro's design aesthetic is based on typography, basic graphics, and simple blocks of colour. The upshot of this is that you can only present data in a way that's not a baffling mess by adding whitespace. It's the only visual cue that you can use to help the user separate the information that they can see in his or her mind. This problem is worsened by the fact that Metro also encourages developers to hide information -- see the previous point.

The fix to this is "don't build apps with low information density". What I said in May is what I'm saying now -- developers need to take inspiration from Metro, but then forge their own path. High information density is not a problem for the average user. It's ridiculous that as a society since the dawn of the digital era we we've been driving up information density. Just because some boneheaded designer thinks Metro looks "way cool" we're suddenly supposed to forget the only point of software, i.e. that the only point that it exists at all is to get information from the screen, through the user's eyes and into their brain. Why on earth any developer would stray beyond that simple premise is beyond me.

I also struggle to understand why Microsoft slavishly went along wit this plan. It must have been obvious to them that low information density would hurt the platform.

"Overly live tiles backfire"

Microsoft makes a big deal about live tiles in Windows 8. For what it's worth, I think it's one Windows 8's best features. I think the opportunity for developers here, especially with line of business apps, is a pretty decent new way of surfacing (no pun) information to the user.

Jakob's point is that a user can configure live tiles in an "unruly" way. Here's an image he offers, which admittedly looks busy.

Here's what he says:

The theory, no doubt, is to attract users by constantly previewing new photos and other interesting content within the tiles. But the result makes the Surface start screen into an incessantly blinking, unruly environment that feels like dozens of carnival barkers yelling at you simultaneously.

What I think developers can take away from this is an awareness that he's probably right and that the garishness needs to be reined in so that it's a bit more Mondrian and a little less Gaudi. Less is more.

Another thing that I've noticed from my personal use is that I really dislike the wide tile format on my Surface, but really like it on my desktop. I wasn't expecting this, so I'd add in the additional point that developers should make sure that both the wide and square tile formats work equally well.

"Charms are hidden generic commands"

I'd suggest that the charms were the part of Windows 8 that he found to be the least good. I also think that in his write-up he's conflated the idea of the charms and the app bar. His central argument -- continually hiding them and having them expose inconsistent behaviour is unhelpful.

Let's start here:

In practice, the charms work poorly -- at least for new users. The old saying, out of sight, out of mind, turned out to be accurate. Because the charms are hidden, our users often forgot to summon them, even when they needed them. In applications such as Epicurious, which included a visible reminder of the search feature, users turned to search much more frequently.

It's interesting to note that as the built-in apps in Windows 8 have gone from beta to RTM, features have been taken from the charms and the app bar into the main app area. In beta, the Maps app didn't used to have a search option in the app bar -- it was only on the charms. In the final version, you can do it from the app bar and the search charm. In the beta version of the Mail app you used to have to summon the app bar to compose an email. In the release version, you always have the button on the main page in the app.

This constant drive to hide features is another feature of Metro that I dislike intensely, and I also railed on this when I wrote about Metro in May. The guidelines ask developers to shove features out of site until their are revealed. Jakob already spoke about the cognitive load from the duality of Windows 8. The insistence that apps hide their options pours oil on the fire.

But as far as this part goes, Microsoft themselves have provided the answer. If the user can't find the stuff they need, put it somewhere where they can. Don't make them think, as Steve Krug said.

In the remainder of this section, Jakob gets close to put doesn't serve the coup de grace on the real problem with charms. They're woefully inconsistent.

Here's one quote:

Furthermore, the charms don't actually work universally because they're not true generic commands. In our test, users often clicked Search only to be told, "This application cannot be searched."

When you look at the rules for getting apps into the Windows Store they're actually quite soft. You don't have to implement search to get an app published. That should be a rule in the guideline. The problem with this as Jakob points out is this: "Enough disappointments and users will stop trying a feature". What this will do is simply block out the feature in the user's mind. It makes that feature globally hard to discover regardless of the installed apps on the system.

The charms problem is more sinister. These are put together in quite a "wooly" way. Windows 8 brokers the sharing requests between the available apps. This is done on the type of data, rather than the meaning. If you have an app that understands how to receive pictures, when the user tries to share a picture your app will be presented. This sounds fine in theory.

And it should work really well, but it doesn't, and I'm struggling to explain why. From a usability perspective it creates a vector for unintended consequences. This whole sharing system is opaque. You can only see the options you get through experimentation. Compare this to drag and drop -- say you want to drag text from a web page and drop it on the mail app to create a new email. Firstly we've understood that paradigm for 30 years. Secondly, the user can see all three players in that operation: the source app, the target app, and the data that's being transformed. How sharing works is that you look at two parts of the problem (the source app and the data), click share and then you get to see your options from the third party. It's easier to "play" with drag-and-drop in this instance. If the mail app can't handle the dragged data, it can pop-up a message. In Windows 8, if the mail app didn't support the feature you get nothing. Likely you'd read and re-read the list of apps looking for it, and then undergo a moment of confusion as to why. You'd then stop trusting the feature, as per the problem with search discussed above.

Given that most of us don't have power to get the guidelines changed, the only thing we can do here is try and do the right thing. Searching you can fix -- every app needs to do something in response to the Search charm. Sharing I'm not sure is fixable. You should absolutely implement sharing where you can. (Apart from anything else, it's a great feature.) I expect sharing to undergo refinement as part of the general process of maintenance on Windows 8.

"Error-prone gestures"

Way back before release when I used to take an old Windows 7 Atom-based tablet around with a beta of Windows 8 on it, everyone I gave the device to would struggle with the gestures. But, during these conversations is that my counter would be that on the iPad, the "four finger swipe up" gesture is hardly intuitive. Gestures tend not to be easily discoverable, but as they are physical, tactile, and visual their multisensory nature means they "stick" very easily. See them demonstrated once, do them once, and you've usually got them.

On my Surface, I've really struggled with the app switching-slash-drop into snapped mode gesture. Snapped mode I think is one of the best UX features of Windows 8 but it's immensely difficult to invoke. I haven't even worked out if you can close an app using touch -- on a desktop you drag the top of the window down to the bottom of the screen with the mouse. The basic top, bottom, and right gestures (app bars and charms access) work perfectly well and are simply.

Parsing Jakob's comments in this context is a little tricky. Firstly:

The tablet version of Windows 8 introduces a bunch of complicated gestures that are easy to get wrong and thus dramatically reduce the UI's learnability.

As per the above, are Windows 8's features any worse than the gestures on iPad? Most of them aren't. The app switching gesture, as mentioned, I feel is very broken. But on the iPad there are many users who have no idea that they have gestures at all. Most iPad users will use the "home" button to get back to the app launcher (do we still call that "springboard"?) and then back into the other app they want.

On this one, I agree with Jakob mostly. Microsoft can fix and improve the broken gestures, if so inclined. But I don't think it matters that the gestures are broken right now because only power users will use them and they'll get used to their idiosyncrasies.

His conclusion

Whilst I think that Jakob is spot-on with the specifics of the issues he raises in his conclusion -- this is essentially a lament on the crazy mixed-up almost half-finished nature of Windows 8, I'm less convinced of what he's saying in his summary. Perhaps the biggest problem with Windows 8 is that it takes time to learn to love it. One of his comments -- "On a regular PC, Windows 8 is Mr. Hyde: a monster that terrorizes poor office workers and strangles their productivity" -- is representative standard stage of grief that nearly everyone goes through when moving to Windows 8. That's not to say he's wrong, but I'm not sure it necessarily matters big picture-wise.

Really, there isn't anything terribly surprising in Jakob's piece. But it is relevant that it's been measured by an expert and reported upon. Of course, Microsoft do their own usability testing. Well, I say that. Microsoft does a massive amount of usability testing, it's just that we don't see the data.

As application developers, Jakob's work is valuable because we now know that, for example, choosing to create a busy and confusing tile for our app is probably unhelpful. We also know that skipping proper and complete implementations of searching and sharing is also unhelpful. I don't think tend to build clickable elements without clearly demarcating as clickable.

But the biggest problem is low information density, so once again it's fair to remind developers that the Metro aesthetic remains a set of guidelines. Your job remains to create software that presents an appropriate amount of information to the user. Don't make the mistake of putting form over function.

What do you think? Post a comment, or talk to me on Twitter: @mbrit.

Image credit: Belinda Mason Photography