Testing times with broadband
commentary Complacency by one Internet provider left them with a poor result in our tests but what if this wasn't a test?
It has been an interesting month of testing at the Lab, where we have revisited a test carried out in 2001 on broadband ISPs. The first test was in the infancy of our broadband testing and at the time we were only able to test a single ADSL 256/64 plan, from a provider using Telstra's infrastructure and Optus Cable. It does not take a genius to guess which of the two services provided the best download and upload performance but you might wonder which was the more reliable.
That is, unless you recall the embarrassing problems Telstra had with ADSL reliability back in 2001. Our testing actually recorded their infamous two-day outage from late Friday afternoon to Sunday evening. Our suite back then was pretty primitive and in ensuing years we were able to refine and expand our test procedures a great deal.
In the past year we have been testing the ISPs registered under the HiBIS scheme for compliance with their contractual performance specifications. It was with this experience we put together a set of test procedures for the ISPs tested in our special broadband review.
As you will notice, they are not represented in the article's comparisons. They were withdrawn at the last moment after it was found there was a fundamental problem with the test equipment installed.
Each vendor was asked to provide an FTP server which was connected via their nominated plan to the Internet; we then carried out the performance and uptime testing with this. In six of the seven cases we achieved the expected results but iiNet's performance was much lower than expected with download and upload speeds of 544Kb/s and 224Kb/s respectively. The upload speed was not too bad when compared to what we thought was the submitted plan but downloads were well shy of 1500Kb -- so imagine our surprise when we were informed it should have been a 12288Kb/s download speed. By this time the two-week test cycle was complete. We received notice two days before testing finished.
It turns out quite a few things went wrong. A technician had connected an ADSL1 modem to the FTP server instead of an ADSL2/2+ modem, and the LAN connection at one end was set to 10Mb/s half duplex while at the other end it was 100Mb/s and full duplex which iiNet says gave it poor throughput. When this was discovered one of the other technicians reconfigured the server and installed an ADSL2/2+ modem, and just to confirm that all was indeed well we ran two quick tests -- a far cry from the 168+ tests run on each previously.
The results were far better with a download speed of 5Mb/s and upload of 672Kb/s. Of course, you would be hard pressed realising the 12288 download speed given the vagaries of the Internet and the long haul from their FTP server in Perth to Melbourne.
I must admit that given the vendor was aware that the results of the test were to be published in a national magazine I would have thought more due care would have been taken with the equipment setup and configuration. If it were me, I would not rest until I had at least confirmed the server was returning the results I expected before it was to be used for magazine testing.
It's lucky our testing requirements had nothing to do with iiNet's typical end-user procedures. I would imagine, for a short while at least, they would have a lot of cranky subscribers if it did.
Steven Turvey is Lab Manager for the independent testing performed for Technology & Business magazine. Send feedback to tandb@zdnet.com.au.
This article was first published in Technology & Business magazine.
Click here for subscription information.