Android/iPhone web browsing speed test flawed

Android/iPhone web browsing speed test flawed

Summary: I'm getting a lot of feedback over a report by Canadian software company, Blaze Software, who have pitted Android against the iPhone in web browsing tests and claim that Android is significantly faster than the iPhone.

SHARE:
55

I'm getting a lot of feedback over a report by Canadian software company, Blaze Software, who have pitted Android against the iPhone in web browsing tests and claim that Android is significantly faster than the iPhone.

Well, you can say that I was curious. So, first off I used the website to run some quick tests, comparing the iPhone iOS 4.2 with the newer 4.3 (for the test I used Google.com as the site to load). I obtained the following results (using the 3 run option):

  • iOS 4.2 - 3.88 sec
  • iOS 4.3 - 2.67 sec

Hmmm, OK, the browser in iOS 4.3 is a bit faster than the browser in iOS 4.2. That's what I'd have expected from my experience with both versions. But that was just one run. I carried out several runs, and what I found was that I got wildly different results, ranging from under 3 seconds to over 7 seconds no matter which iOS version I used. It seems that the tool is highly susceptible to variation.

I then decided to do some more digging and read Blaze's blog post a little bit closer. I was curious as to how the test was done, in particular how the data was being extracted from the the browsers (especially Safari on iOS, since iOS is a closed box). I came across the answers I was looking for:

The measurement itself was done using the custom apps, which use the platform's embedded browser. This means WebView (based on Chrome) for Android, and UIWebView (based on Safari) for iPhone. Manual verification showed that page load performance of the embedded browsers, when properly configured, is effectively identical to the stand-alone browsers. The load times are calculated using the "Document Complete" callback from the browser, which is a standard way of measuring a web page's load time. As mentioned above, the agents are now a part of a free service available at http://blaze.io/mobile/, and we encourage you to try it out.

Hold on a moment. Blaze is using a custom app. There's a problem with that. As I talked about the other day, Apple is limiting the use of the improved Nitro engine in iOS 4.3 to Safari. Other apps that use the UIWebView controller used by applications to access the web. This could well skew the results significantly.

Given the UIWebView controller issue, combined with the wildly fluctuating results I was getting for one site, I'm going to say that all data puled from this test should be treated as speculative and for entertainment purposes only.

But, even if we assume that the tests are 100% accurate for a moment, is an average load time difference of 1.1 second really going to make that much differenc eis the overall scheme of things when you're dealing with huge variables such as 3G and WiFi connections?

[UPDATE: In the time I was writing this, other have weighed in. Here's what The Loop has to report:

"Their testing is flawed because they didn't actually test the Safari web browser on the iPhone," Apple spokesperson, Natalie Kerris, told The Loop. "Instead they only tested their own proprietary app which uses an embedded web viewer that doesn't take advantage of Safari's web performance optimizations. Despite this fundamental testing flaw, they still only found an average of a second difference in loading web pages."

 It would be a good test ... if it weren't flawed!]

Thoughts?

Topics: IT Employment, Browser, CXO

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

55 comments
Log in or register to join the discussion
  • RE: Android/iPhone web browsing speed test flawed

    Waaaaaa, Apple didn't win. Waaaaaa. I'm surprised the Apple people were awake to respond, must not use an iPhone for an alarm. And with the antenna issues, they must have replied from a more reliable connection.

    :)
    timspublic1@...
    • RE: Android/iPhone web browsing speed test flawed

      @timspublic1@... go back under your bridge you whinny little troll.<br><br> The test was not even run on an iPhone, it was run on an emulator.<br><br>
      And on the emulator it still only show the difference of 1 second in load times (on average). I, for one, along with many other will most likely not fret over a whole extra second of waiting for a page to load.
      Geuseppi
      • And Anadtech made also extensive test of iOS 4.3 on iPhone 4/iPad/iPad 2

        and found that the browsing speed is about no less than on 1GHz Androids or much better (though this is for iPad 2).
        DDERSSS
      • Only thing flawed is the iPhone

        One second compounded over thousand pages visited during the lifetime of a smartphone is a lot of wasted time.<br><br>But then its an iPhone, and some of us know better. Shiny is not always best. Its just shiny and not useful.<br><br>I dare Apple prove that the "non-flawed" test would improve its product performance to the point it winvs over Android. Technical people know it wont matter what they do. The hardware is inferior, the browser is as well, consequently the obtained results. <br><br>Now on looks, you have to give Apple credit. But what if its shiny on the outside and inferior on the inside.. you can't hide.
        Uralbas
      • RE: Android/iPhone web browsing speed test flawed

        @denisrs actually cbsnews was considerably faster on Android in those tests and honestly, when that Android tablet gets 4G the whole test will be significantly slower on the iPad 2 regardless of what version of the OS they are running.
        slickjim
      • RE: Android/iPhone web browsing speed test flawed

        @Uralbas Your small minded bias aside you might try looking at details. If the test doesn't utilize the built in optimization then the test is flawed and there is nothing your hatred for Apple can do to change that. By the way, your "lot of wasted time" with the extra 1 second each for a thousand pages comes out to what, less than 1/2 hour over the lifetime of the phone? Since there is a faster app for pretty much everything I want to do with a browser I'm not going to lose that supposed 1 second but that's all right, you continue with your stupid drivel so you can feel better about yourself and your choices.
        non-biased
      • RE: Android/iPhone web browsing speed test flawed

        @Peter Perry If most of the units being sold are WiFi only and used on a WiFi network how exactly is the 4G going to be a factor if and when it ever does get the upgrade?
        non-biased
    • RE: Android/iPhone web browsing speed test flawed

      @timspublic1@... Face facts there trollboy - the test that made Android seem like it was way faster than iOS was flawed... get over it.
      athynz
      • RE: Android/iPhone web browsing speed test flawed

        @athynz I don't think it was flawed. Apple makes the claim that "it's for security" but that claim is ludicrous since if a webpage is launched from an icon, iOS has COMPLETE control over the app... etc...

        so, something stinks here!
        mrminnman
    • RE: Android/iPhone web browsing speed test flawed

      @timspublic1@...

      Watch out Tim, you upset the Apple fanatics. I hope you girls enjoy your Apple products.
      Rob.sharp
      • RE: Android/iPhone web browsing speed test flawed

        @rob.sharp@... Thanks, I am enjoying my Apple product very much and have never had a second thought about getting it :-)
        non-biased
  • RE: Android/iPhone web browsing speed test flawed

    It is a sad day for an OS when someone needs to "cheat " on a performance test to generate bad test results for the OS that they do not favor.
    Geuseppi
    • As far as I've read Blaze.IO had no preference...

      @Geuseppi

      ...over either OS. They just ran a test they thought was fair, and it turned out that it wasn't.
      DevJonny
  • RE: Android/iPhone web browsing speed test flawed

    Ah! The first FUD of spring from the Apple faithful.
    What a ridiculous comparison. No IT Professional worth their salt would subscribe to this conclusion.
    If you can't beat them - spread fear, uncertainty and doubt (FUD).
    HockeyTown4Ever
    • RE: Android/iPhone web browsing speed test flawed

      @HockeyTown4Ever

      Good to know that MS would never do such a thing. "getthefacts" as one example was simply the truth, the whole truth and nothing but the truth . . . NOT!
      Wakemewhentrollsgone
    • RE: Android/iPhone web browsing speed test flawed

      @HockeyTown4Ever : has anyone still have any doubt that Adrian H-K is a Apple-paid blogger?
      nomorebs
  • RE: Android/iPhone web browsing speed test flawed

    What is being tested here? If you're looking for the loading speed of webpages then the fact that you're using "Views" should matter. Apple designed the UIWebView, just as Google Designed the WebView. If Apple limits their views, which I think is easier to implement on iOS, then its their fault. And I will say that from my experience programming on both platforms Google get my vote. Besides, since when could Safari ever stand up to Chrome? One thing I would like to know is if the Nexus was using vanilla, or another ROM. This could skew results.

    And For the Emulator comment. The iOS emulator is far better than Androids... There isn't any cheating. Apple is just upset they aren't top dog.
    Cjconry
    • RE: Android/iPhone web browsing speed test flawed

      @Cjconry Right... What IS being tested here?

      It's certainly not the SAFARI browser, because that's not what the testers were using. They were using the embedded viewer, because they chose to use a custom app.
      lelandhendrix@...
    • RE: Android/iPhone web browsing speed test flawed

      @Cjconry And by the way--you know that Google is using WebKit for their browser, right? The same WebKit that Apple invented and gave them for free?
      lelandhendrix@...
  • RE: Android/iPhone web browsing speed test flawed

    Typical Apple a-holes. Any criticism is wiped off the board.
    Good luck listening to yourselves in your echo chamber.
    HockeyTown4Ever