Optimum Ultra: So how fast is it?

DOCSIS 3.0 Systems Architecture (Source: Cablemodem.com) Click on the diagram to enlarge.

The on-site visit is required because in order for the service to function properly, only a minimal amount of RF noise can be on the coaxial cable line, so typically the premises cable and connectors are swapped out and the line run to the local "Node" is tested for signal quality. Each Cablevision distribution node services approximately 500 customers, and connectivity from the node is home run via fiber-optic cables back to the main Optimum collocation centers.

In most circumstances, the visit should be a half hour or less to swap out the modem and check the signal quality. In my case, the supervisor elected to replace my entire cable infrastructure, and do some maintenance on the street-level signal amplifier that was mounted on the telephone pole which was having some moisture-related issues with its physical housing, so it was a 3 hour visit. I appreciated his thoroughness and attention to detail, so I have no qualms with the length of the outage.

After the modem is installed, the technician runs a series of speed tests using OOKLA's Speedtest.net site, using a direct CAT-5 connect to the DPC3000 from his laptop, which has a 1000Base-T (1Gig Ethernet) connection. On the first run, he got over 112Mbps using a server connection that had 21ms latency which was less than 50 miles away on Cablevision's own network. Satisfied, he went on his way, and I connected the modem to my router at the time, a NETGEAR Prosafe SRXN3205, which is an upper-end, branch office 5GHz wireless VPN rules-based firewall router with 8 gigabit Ethernet LAN switch ports, and a gigabit Ethernet WAN port for connecting to DSL, cable modem or other network equipment.

PDF Download: optimum-online-speed-tests-ookla

The architecture of my home network consists of 2 NETGEAR GS608 v2 8-port gigabit Ethernet switches, both of which host several desktops and test servers in my office and are directly attached to the broadband router. In addition to the switches the broadband router is connected to a VOIP appliance and a dedicated NETGEAR 5Ghz 802.11N wireless bridge that is linked to two other 5Ghz 80211.N bridges, one of which resides in my wife's office and provides her PC and printer with connectivity.

The second bridge provides 270-300Mbps connectivity for my Slingbox and my Roku Netflix streaming device in my living room. Additionally a single 2.4Ghz Linksys Wireless-N game adapteris connected wirelessly to the router for my DirecTV DVR in my upstairs bedroom. All the NETGEAR bridges are forced into 5Ghz Wireless-N mode so that there is no mixed traffic coming in from these devices and are dedicated to high-bandwidth tasks such as downloading movies. Every PC or server which is wired to a switch is forced to 1Gbps Full. Suffice to say this type of high-speed home network is exactly the type of customer that Optimum Ultra is designed to service.

So as soon as the Cable Guy left the house, I jack in the SRXN3205 and start doing some tests, with only my desktop PC switch connected so that my XP laptop and test Windows 7 PC have connectivity. The first thing I noticed was that on every single OOKLA speed test that I did to local NY-based, low latency servers is I maxxed out at about 50Mbps.

I had thought that Cablevision had pulled a fast one on me, but sure enough, when I direct connected the modem to my laptop or the Windows 7 test box, I got much higher transfer rates, around 100 megabits. This led me to suspect that either the SRXN3205 was misconfigured or some type of protocol negotiation issue or packet loss was occurring.  I'm not entirely convinced that the SRXN3205 is a bad solution, but I haven't had the chance to follow up with NETGEAR yet to see if we can resolve it -- I had my installation on July 3rd, and had little time to resolve the problem before everything shut down for the holiday weekend.

EDIT: Here's what I heard from the NETGEAR product manager, Jason Leung:

This is because the SRXN3205 is only rated 60 Mbps!  A firewall is very different from a router and you can’t compare the throughput between the two.

This is because a Router is not a Firewall.  A Router routes every single packet as fast as it can.  A Firewall has to do real work to determine which packets to route and which to drop ---- hence it’s a VPN/FW, not a router.

A lot of consumer grade equipment advertise FW capabilities but in reality all that consumer grade equipment is doing is NAT.

For customers which need a true rules-based VPN firewall for Ultra or FiOS, Jason recommends the FVX538, which has a WAN port that is rated for 100Mbps sustained data transfer, as opposed to the SRXN3205.

To eliminate the possibility of it being the NETGEAR router, I went out to Best Buy and purchased a new Gigabit SOHO router, an Apple Airport Extreme. Sure enough, I got very high throughput with the OOKLA tests with the Airport Extreme, and I probably would have kept the device had it not been for the fact that the Extreme has very limited firewall capabilities and lacks port triggering and a number of other advanced networking features I need for various types of testing.

The Airport Extreme also lacks a web-management interface, and only has 3 switched Gigabit Ethernet ports for LAN attached devices, which I think is a bad design choice for what is probably the most expensive SOHO router on the market. In my opinion it should have a web GUI and more advanced firewall functionality and at least five ports for $179.00. However, if you are a SOHO user that happens to use a bunch of Apple products such as the Apple TV and Macs, you'll probably love the product, particularly if you are using a service such as Ultra or FiOS. You'll also notice from the speed test PDF that is enclosed in this post that it scored overall the fastest in the 3 routers I tested with on the OOKLA benchmarks.

In any case I returned the Airport Extreme to Best Buy and decided to go with another product, the CISCO Linksys WRT610, which is a gigabit Ethernet 5Ghz/2.4Ghz dual band SOHO router. My rationale with going with a Linksys instead of another NETGEAR was that I wanted to eliminate any possibility of there being a vendor compatibility issue between the CISCO and NETGEAR equipment. In theory, the WRT610 should be fully optimized and certified to work with CISCO's own broadband equipment even though Scientfic Atlanta and Linksys are two different subsidiaries.

Over the 4th of July weekend and through early this week, I ran a battery of OOKLA speed tests on Speedtest.net to determine just how fast the quality of service I was actually getting. OOKLA's methodology for SpeedTest.net is documented on their web site, but suffice to say that the lower the latency you have to the target server, the faster you are going to go. So anything under 20ms ping times you are going to get very fast results.

As you increase packet trip times, your speed decreases dramatically. Trip time is entirely dependent on the speed of the link at every step of the route and how many "hops" you have between your premises equipment, to the backbone of your broadband provider, out to the Internet, and to the target server. You also have to assume that with any network connection that a SOHO router itself is going to add packet overhead, around 10 percent, and you'll see another 20 percent overhead on the WAN connection itself just added by using TCP/IP itself. So 70 percent utilization is really the best you can hope for, in fully optimal circumstances.

That being said, OOKLA's SpeedTest.net site is not necessarily a good overall test of whether or not you can saturate the broadband connection, because it only does about a 10-20 second burst of data from one end to the other. If you are connecting to a low latency server, that is barely enough time for the transfer to "Ramp Up".  Therefore I decided to do more extensive testing and "Loaded the deck" for a best-case scenario to max out the line.

For my first series of speed tests, I contacted Limelight Networks to see if they could provide me with some large files I could download over http protocol using Linux's wget utility, which is the web protocol equivalent of an FTP client. Limelight, as some of you may recall, is a large content delivery network provider and was the hosting solution that NBC chose for delivering the Olympics video content. They were happy to oblige, providing me with a set of files that were within 9ms ping time from my premises equipment, and less than 3 hops away from Cablevision's network.

PDF: limelight-pingresults

PDF: limelight-bigfiles-withrouter (100 and 500MB file download using WRT610N)

PDF: limelight-modemonly (same tests as above with CISCO modem direct connected to PC)

Note: All tests above use obfuscated results to mask identity of the servers used to protect content and supplier bandwidth. All metrics are unaltered.

As you can see from the test results, when downloading from Limelight, the transfer speed was maxxed out at around 32Megabits or around 4MB per second, way short of the full 101Mbps that is advertised. I'm still not finished doing tests with Limelight and this slower than expected speed may due to a QoS configuration issue, so continue to watch this space. [EDIT: See "Trafficking in Michael Jackson" for an explanation. We got Jackdotted.]

As a second test, I contacted a friend who owns a large ISP based out of Boston, who has a collocation center with 3Gigabits of aggregate bandwidth. His servers are less than 20ms ping from my premises, so I did some large file download tests using the same methodology as with Limelight. I did a lot better, around 8.5 to 10MB per second. The tests were performed at off hours, at 10-11PM at night, where the least amount of traffic was hitting his servers, so this was a purely best case scenario where latency was as low as I could expect and incoming bandwidth was as big as I could possibly want.

PDF: 3gigpipes (ISO file download using WRT610N)

[EDIT: A re-attempt at the Boston-based 3gig provider on July 9th yielded a burst transfer rate of 11.4MB per second, or 91Mbps on 579MB ISO]

The results match with my expectations of 20 to 30 percent overhead on the bandwidth with an expected net performance of about 70-80Mbps (10MB a second) under very optimal conditions.

In summary, I do think that Ultra and other high-bandwidth services such as FiOS do provide unparalleled speeds to subscriber households, but the caveat is that you cannot expect to get performance like this all the time, and that network overhead, latency and routing is a reality we're never going to get away from. As more content ends up at CDNs like Limelight and Akamai where bandwidth-intensive files are locally cached near the customer, the benefits of these fast broadband connections will be more evident -- but until then, early adopters will have to put up with vastly fluctuating performance.

What have your experiences with Optimum Ultra and Verizon's FiOS been like? Have you performed similar benchmarks? Talk back and let me know.