User Centric iPhone study proves, well, nothing really

User Centric iPhone study proves, well, nothing really

Summary: I'm still trying to figure out what usability consultancy group User Centric are trying to prove with its usability study of the iPhone's keyboard compared to QWERTY and numeric phone keyboards.

SHARE:
TOPICS: iPhone, Mobility
21

I'm still trying to figure out what usability consultancy group User Centric are trying to prove with its usability study of the iPhone's keyboard compared to QWERTY and numeric phone keyboards. 

Let me start off with the most glaring flaw as I see it with the study [emphasis mine]:

A total of 20 participants were brought in for one-on-one usability sessions with a moderator.

20?!?  No matter how you cut it, that's a small sample. 

All sent text messages at least 15 times per week.

So these people aren't heavy users, sending just two or three messages a day, depending on whether they're a five-day-a-week texters or seven-day-a-week texters.  I don't send many text messages in a week but I send a lot more than that.

Ten of the participants owned a phone with a QWERTY keypad, and ten of the participants owned a phone with a numeric keypad.

OK, pretty even split, but ...

Those who owned a numeric keypad used the "multitap" method of entering text messages rather than predictive text. To multitap, a user must press a particular key on the numeric keypad multiple times to get the desired character to appear.

So these people don't use predictive, so they're not the brightest of the bunch.

None of the participants were iPhone owners ...

Well duh!

Although participants were given one minute to familiarize themselves with the iPhone's touch keyboard, their texting abilities on the iPhone were still at the novice level.

Sheesh, one minute.  Who's ever grabbed a brand new phone and started sending messages after just taking a minute to familiarize themselves with the keyboard? 

Throughout the study, we did notice limited improvements in keyboard comfort as users progressed through the tasks on the iPhone.

This is a throwaway statement but it's valueless because we don't know how long, on average, each user had with the iPhone.

Here's another loaded statement:

In general, participants took longer to enter text messages on the iPhone than on their own phone. Despite the keyboard similarities, QWERTY phone users took nearly twice as long to enter comparable messages on the iPhone compared to their own phone. On the other hand, multitappers did not experience a significant difference in the time it took them to type messages on the iPhone.

Hmm, wait a minute.  That statement starts off by saying that participants took longer to type on the iPhone that their own phone and then goes on to say that the multitappers didn't experience a significant difference in the time the exercise took.  Anyone else feeling a bias in this study?

Participants also made more typing errors on the iPhone. This phenomenon was expected since users had much more experience with their own phones.

Again, so what?  What would have been interesting would have been to also give a phone with a QWERTY keyboard to the multitappers and a phone with a numeric keyboard to the QWERTY users and see how they got on with that.

My personal take of the study is that it's next to worthless.  It doesn't really prove anything of value beyond:

  • If someone doesn't pay for a gadget they're not invested in learning how to use it
  • If you don't take the time to learn how to use something, then you can expect to have more trouble than someone who does take the time

If you're interested in reading more about the study, you can find it here

Thoughts?

Topics: iPhone, Mobility

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

21 comments
Log in or register to join the discussion
  • They're trying to get publicity

    An awful lot of polls & studies are simply done to generate news coverage for the sponsor. Trust me here. I've written questions for client polls that are then conducted by polling outfits.

    Did you ever hear of UserCentric before? You have now. Mission accomplished.
    j.m.galvin
  • keypads are tricky; iphones are impossible for visually impaired

    What if you are blind? I have friends who are, and they have to spend $1500 on custom screen reading software to be able to make Windows/Mac screens readable. Dont'know what they do for hearing text messaging anyway, but I don't see any way to make this kind of touch screen technology easily accessible; does anyone else? Keypads and buttons can be memorized and be made to speak--how does one do that with a touch pad screen with nothing anchored down?
    klassman6
    • I can't imagine

      why someone who is visually impaired would spend that kind of money for the features that the iPhone offers anyway. Most of its key features are, well, visual.
      Michael Kelly
      • it's the future that's at stake

        I'm worried about future options, not current choices. Visually impaired folks were fine with DOS, but when Windows came along, they had to locate icons and windows on the screen, something that is still a huge challenge. My concern is that with virtual keypads and touch screens in vogue, the keypad will go the way of the cursor line and folks with VI will find themselves left in the dust again
        klassman6
        • The future may very well include voice commands

          Are we talking VI or blind here? I imagine that an ICON in most cases is larger and
          brighter than text. At least that has been my experiance. Half the time I can't make
          out words on a screen because the resolution is set so hight but I can still make out
          an icon on the desktop.

          Pagan jim
          Laff
          • I have the solution to your problem!!!

            [i]Half the time I can't make out words on a screen because the resolution is set so hight[/i]

            The problem is that OSX's font rendering is so bad that it blurs fonts and makes them too faint to read.
            [url=http://blogs.zdnet.com/Ou/?p=682] Vista puts Mac OS X font rendering to shame [/url]

            Switch to Windows or Linux and you will find everything so much easier.

            You're welcome. :)
            NonZealot
          • Yeah hitting my face with a shovel will take my mind

            off the "itch" on my foot to.....wonderful solution there NonZ...:P

            Pagan jim
            Laff
          • The real problem is for blind folks

            Doesn't matter how bright something is if your vision is zero. Keypads orient you so you can input information, and screen readers like Jaws for Windows still depend on keypad input to be able to select an icon or window to read. What happens when the keypad goes virtual and there is no way to locate it if you are blind? Seems like a real nightmare for accessibility issues.
            klassman6
          • Blind people don't matter to Steve Jobs

            He isn't blind so why should he care about their difficulties? Remember, buying into the Mac ecosystem is all about aligning your requirements to Jobs' requirements. Abandon all hopes of thinking for yourself or believing that your wishes are the least bit important. There was a hilarious, HILARIOUS [url=http://www.youtube.com/watch?v=H3q8_40GBuI] Mac Rant [/url] which contains the best quote I've ever heard about what it is like to work with a Mac. It was basically along the lines that the Mac does whatever it wants to and if what you want to get done happens to align itself with what the Mac feels like doing, all the better!
            NonZealot
          • Good call it is but a RANT.....that is the height of it's

            achievment nothing more. The iPhone like the Mac and OSX are but choices if it fits
            and you like it....you are free to choose it. If it does not for what ever reason there
            are other choices out there like Linux and Windows. So again where is the problem?
            Same for the iPhone it is but a choice no one is forced to use it and there are plenty
            of choice available right now and for the forseable future...again "Where's the beef?"

            Pagan jim
            Laff
          • Another stupid complaint

            You're not making sense complaining that Jobs isn't designing this device around
            blind users, when no mainstream text-messaging devices do that.
            Do you have any sense of what is an appropriate criticism? So far, you seem to be
            spouting off at anything a small mind can invent.

            It was Jobs who made sure the Mac had better user-assistance features and
            compatibility than any other computer has ever offered. That includes visual
            impairment. You clearly have never looked.
            maxsnorkel
          • Two questions....Voice control?

            What about an Keybad add on by a third party vendor? Can't the iPhone take such I
            don't know myself but I would think it might be possible. No I don't think the keypad
            is going to disapear anytime soon do you? I mean look at all the phone models out
            there right now do you think they are going to disapear anytime soon and in the
            mean time I'm surre voice control will become an option.

            Pagan jim
            Laff
  • Just a comment regarding predictive txt.

    I'm pretty bright, but I really, really hate it. I'm much faster using the old multi-tapping than predictive text - most of the time it never predicts what I wanted correctly anyway.

    The only time it predicts correctly when it's a word I've used often, and that's never much faster than tapping it out - I usually finish the word just as I notice it's predicted it for me.
    SniperCT
  • What do you know Adrian?

    You are just another Apple lover who hates everything about Microsoft. You are so biased and my opinion of you will never change. I'll just choose to ignore all the blogs that have been critical of Apple because, well, that would mean that you don't fit into a nice little neat category and blurry categories kind of confuse me.

    ;)
    NonZealot
    • You confused!?! Naw....tell me it aint so!!!

      Pagan jim
      Laff
    • That's stupid; you can't discard the meaning

      It's irrelevant whether you find the author important, but obviously you are lying. You
      came to the article to read it, you stayed to reply -- so you don't know how you feel.

      It doesn't matter how you fell about the author -- the only meaningful criticism is
      about his arguments. If you can't contradict an argument, he's entirely correct. (And
      you should probably change your attitude!)
      maxsnorkel
  • Adrian - learn statistics

    Before you open your mouth to put your foot in it, why not learn what sort of populations are necessary for the research. 20 is more than enough for this kind of study.
    tonymcs@...
    • Princess bride quote:

      I don't think that word means what you thing it means" You've used this a great deal
      the whole "Learn statistics" and claim by default that you know what you are talking
      about but might I ask that you provide some sort of proof to this otherwise it seems
      you very well might be a troll or ill informed youself.

      Pagan jim
      Laff
    • Actually no...

      20 can be a proper sample size, depending on what you are studying. However that assumes a large sample population available to draw from. We have no way of knowing how they selected their population or if it was random. As far as we know they selected the 1/2 of the 20 from RIM headquarters which may impact the results a tad.

      Besides... dualing text messages on cell/smart phones... after a few minutes of familarization? Not exactly the most important question to cover out there. Almost as important as what OS my e-mail client needs to run under.
      Jim888
  • Left the biggest issue out

    Adrian, I think you left out the very biggest issue -- that the experience in the
    first few minutes is not relevant to whether it will be a useful device or
    improvement.

    Those users got more than a minute to practice with keypads originally, and every
    user has a LOT more experience with QWERTY keyboards that can be pecked at.

    A fair study of the practical use of the iPhone input would ONLY have compared
    people who had been using both iPhone and one of the other types for enough
    time to be comfortable with both. That shows the difference in the input
    performance.
    What this study tried to show is ONLY how quickly a non-owner would be able to
    send a message without any experience. A nonsense study entirely.
    maxsnorkel