X
Tech

The customer is not always right

How many times have you heard the old adage, "The customer is always right"? If you're like me you just took it as an axiom, something that was beyond questioning.
Written by Ed Burnette, Contributor

How many times have you heard the old adage, "The customer is always right"? If you're like me you just took it as an axiom, something that was beyond questioning. I think it's time for software developers and project managers to start questioning it.

Now, I'm not talking about the typical retail situation where you take your case of Diet Coke back inside the store, trying fruitlessly with your fingers to stem the cola fountain that exploded when the case fell out of the shopping cart in the parking lot and struck the pavement with disastrous and messy consequences. That was obviously the fault of the store clerk that packed the cart and didn't take into account simple principles like gravity and friction when stacking the cases above the top edge, ready to become airborne, if momentarily, at the slightest deceleration.

Instead, I'm talking about that stage in software development where you're trying to determine what you're supposed to be writing and what it needs to do. How many users should it support? How much data should it handle? Little things like that.

The most natural approach would be to ask the customer. They should know what they want, right? And wouldn't it be nice if you had some nice neat directions written down in a "requirements document". This will help you all "manage expectations" later, so you can point out to the customer what it is they should be happy with. Oh, think of all the meetings you can have, working out the details of these requirements. Think of how good everyone can feel about it when it's done, how the project leaders can pat themselves on the back for hammering out all the issues. Sadly, there's one little problem with the results you get...

It's fiction.

For the most part, the "requirements" will be made up. As in pulled out of the air. Bearing almost no relationship to reality, created from whole cloth. As with any rule there are exceptions, but speaking for myself, I've generally found this to be true. Not only can it be useless to you, it can be harmful, causing you to expend resources doing things you didn't have to do, and skipping things that were really important. It can also be harmful to the user. Even if you give them everything on the list, they may still be unhappy, because satisfying a list isn't what they wanted. It's not their end goal for using your product.

Other industries understand this. Do TV networks simply ask people what their favorite shows are and what they want to watch next? No, they hire Nielson to put devices on thousands of TVs that measure what people are actually doing. They put out new shows, keep the popular ones, and can the others. Do racing teams work out ahead of time exactly how fast their car needs to go? No, they just want a car that can win the race.

If you ask a customer if they want features A, B, or C, more likely than not they'll say "yes". Why not; it doesn't cost them anything and they can see how it might come in handy later. Then they'll go back to the office and grump to themselves about how awkward it is to fill out a form, or how long it takes to process a days worth of data. Or worse, they may not even realize that things could be better, and they may be leaving opportunities (in terms of productivity, better decisions, lost income) on the table.

Luckily there's a better way: Watch them and feel their pain. Listen; not so much to the details of what they say they want, but to the underlying causes. What are they trying to achieve, and how is your software helping and hindering them to achieve that?

Challenge all your assumptions, and question all your feedback. A powerful tool you can use to do this is iterative development, deployment, and measurement.

Marissa Mayer, VP of Search and User Experience at Google recounts a story of how users requested more search results on a page. The default was 10, and many users said they wanted 15, 20, 30, or more. Where did 10 come from? When Google was first created, it was the number of results returned by another search engine, AltaVista. So it was arbitrary, and this request came up over and over, so they were tempted to just change it. But instead of bumping the number and moving on, they decided to test out if this is what people really wanted.

First Google came up with a way to measure user satisfaction, for example if people were finding what they wanted and clicking on the results. There are several different criteria that they keep track of for just this purpose. Next, out of all the searches people did over a certain period, for some percentage they made it return 10 results, for another percentage they made it return 20, and so on. They call this "Split A/B testing" even though there could be more than two choices. After a short period of time (given the volume at Google you can have good results in minutes), they found that users preferred 10 results. And it wasn't just by a little bit; click-thru and future searches went down significantly when more than 10 results were shown. It was completely counter to what the users said they wanted.

As my favorite TV doctor likes to say, "Everybody lies." He doesn't even talk to his patients if he can help it. While I wouldn't go that far, the symptoms your customer is trying to alleviate by coming to you, the data volumes they are dealing with now, the number of users they have, the deadlines they have, and how the product is performing right now in the field, all these are more telling than anything you'll find in a "requirements document".

Don't "manage expectations" -- deliver results.

Editorial standards