Level 3 heats up the Netflix, Verizon internet war

Level 3 heats up the Netflix, Verizon internet war

Summary: Level 3 accuses Verizon of admitting to deliberately slowing down Netflix traffic.

SHARE:

Mark Taylor, VP of Content and Media at Level 3, a global Tier 1 business and consumer internet provider, has accused many end-user ISPs before of slowing down Netflix traffic.

Now, Taylor's gone further. He's accused Verizon of deliberately slowing down Netflix customers' videos.

Level3-Verizon Interconnect
According to Level 3, Verizon itself shows that it streams Netflix video slowly because it's too much of a cheapskate to pay for more 10Gbps ports at its internet Tier 1 interconnect routers.

Taylor states that Verizon has itself revealed that it's screwing Netflix customers over.

Taylor wrote, "David Young, Vice President, Verizon Regulatory Affairs, recently published a blog post, suggesting that Netflix themselves are responsible for the streaming slowdowns Netflix’s customers have been seeing. But his attempt at deception has backfired. He has clearly admitted that Verizon is deliberately constraining capacity from network providers like Level 3 who were chosen by Netflix to deliver video content requested by Verizon’s own paying broadband consumers."

The proof that Verizon is lying, according to Taylor, can be found in its diagram. It implies that the Verizon "network has lots of unused capacity at the most busy time of the day." In other words, "Verizon has freely admitted that it has the ability to deliver lots of Netflix streams to broadband customers requesting them, at no extra cost. But, for some reason, Verizon has decided that it prefers not to deliver these streams, even though its subscribers have paid it to do so."

He pointed out that Verizon portrays Level 3 is the bottleneck in the internet. Taylor claims that simply isn't true.

In an earlier blog post, Taylor stated that Level 3 has more than enough bandwidth to support its internet interconnect partners, but that some ISPs, such as Verizon, "have left interconnection ports congested and deliberately harmed the quality of the services that customers have paid them for."

So why does Verizon blame Level 3 and the other Tier 1 network operators for Netflix traffic problems? Taylor says that it's because Verizon refuses to upgrade their interconnect points.

Taylor specifically cites the example that Verizon picked: Verizon's Los Angles Network Operations Center (NOC). "All of the Verizon FiOS customers in Southern California likely get some of their content through this interconnection location. It is in a single building. And boils down to a router Level 3 owns, a router Verizon owns and four 10Gbps Ethernet ports on each router. A small cable runs between each of those ports to connect them together. This diagram is far simpler than the Verizon diagram and shows exactly where the congestion exists."

The solution? With no congestion on the Verizon or Level 3 sides, "We could fix this congestion in about five minutes simply by connecting up more 10Gbps ports on those routers. Simple. Something we’ve been asking Verizon to do for many, many months, and something other providers regularly do in similar circumstances. But Verizon has refused. So Verizon, not Level 3 or Netflix, causes the congestion. Why is that? Maybe they can’t afford a new port card because they’ve run out – even though these cards are very cheap, just a few thousand dollars for each 10 Gbps card which could support 5,000 streams or more. If that’s the case, we’ll buy one for them. Maybe they can’t afford the small piece of cable between our two ports. If that’s the case, we’ll provide it. Heck, we’ll even install it."

What really proves the case that Verizon and other ISPs are deliberately throttling Netflix videos, according to Taylor is that "This congestion only takes place between Verizon and network providers chosen by Netflix. The providers that Netflix does not use do not experience the same problem. Why is that? Could it be that Verizon does not want its customers to actually use the higher-speed services it sells to them? Could it be that Verizon wants to extract a pound of flesh from its competitors, using the monopoly it has over the only connection to its end-users to raise its competitors’ costs?"

Verizon has not replied to a request for comment. Lately, Level 3, Netflix, and Verizon have taken to communicating by dueling blog posts. 

That said, Taylor's position makes sense to me.

Yes, getting higher bandwidth to the last mile is expensive. ISPs are looking to newer technologies such as G.fast DSL and DOCSIS 3.1 to bring gigabit speeds to the home without the high costs of laying out fiber.

To improve the speed at the junction between the ultra-high-speed Tier 1 ISPs and the consumer and business ISPs is way cheaper. It really doesn't require anything more than adding 10 Gbps ports at the interconnect routers. The technology is here, the switches and cards are available for reasonable prices.

The only real reason I can see for Verizon, and other anti-Netflix ISPs, to continue to slow down traffic is because they want to break net neutrality to increase their own profits at the expense of video providers and their own end-users.

Related Stories:

Topics: Networking, Broadband, Verizon

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

19 comments
Log in or register to join the discussion
  • Reason For Bottlenecking the Connection

    its due to them being providers of this service as well. lets make it slow enough so you get fed up with Netflix. so they will be questioned into using their services and thus or alas you can buy their services for use like DirectTV or FiOS TV and services. thus making it easier for them to charge you more money for these services than what Netflix (basic internet connection (10 megs minimal) required) charges for unlimited streaming content. heck even Hulu combined with netflix is cheaper than my Comcast TV services. put them together and now you will see why they do it.

    is it wrong and fraudulent? Yes. can this be put into a law suit? Yes. but make sure you read the terms of your contract. they may have put something in there that states limiting (throttling) traffic after a certain size or gigabit amount has been met or what have you. which lots of companies sadly do and or used to do in the past.
    TestTatter
  • Is all this in the submissions to the FCC???

    I don't need to be reading this in the press if that flunky Wheeler and his voting majority of fast lane advocates on the FCC aren't forced to deal with it formally in their decision. The Level 3 type providers are all working under net neutrality and apparently happy with it, it's the last mile Verizon ISP types that want to sell speed lanes to the 1% and are picking fights with content senders like Netflix. The FCC should not be changing the rules on the basis of undecided facts, repeat FACTS, about who and what is clogging the streaming services.
    I2k4
    • FCC commissioners follow the news too

      They don't have to hear about conditions related to their duties in formal hearings alone.
      John L. Ries
      • it is not a lack of awareness

        on the part of the FCC. The question is is will the be FCC more responsive to the public interest or to lobbyists with deep pockets who want to tilt the playing field.
        krossbow
  • Router ports not the issue balanced peering agreement are

    I've been following the Netflix/L3/Verizon finger pointing match and from what Verizon said, the issue sounds like L3 is not willing to pay Verizon for the unbalanced peering traffic.

    Classically, two ISPs, like L3 and Verizon, agree that if the traffic being sent in each direction is about equal, it's silly to bill each other the same amount. If one party is sending more traffic, they need to pay the other ISP to deliver that traffic to it's destination. Verizon has stated that the peering agreement with L3 IS a balanced peering agreement, and that traffic flow is NOT balanced, probably due to Netflix.

    What L3 is suggesting is Verizon should add router ports to make the flow even more unbalanced. My guess is Verizon is happy to do so, provided L3 pays for their share of the traffic delivery costs.

    So L3, here is you chance to clarify the situation. Is the peering agreement a net zero cost balanced traffic agreement? If not, are you willing to pay the customary charges for delivery of your excess traffic? Or do you believe Verizon is asking an unreasonably high fee that's significantly different than the norms? To me it sounds like L3 wants Verizon, and indirectly their customers, to foot the bill for the unbalanced peering traffic. I assume L3 does not allow traffic to flow into their network for free, unless an equal amount also flows to another peer.

    I could also imagine L3 and Verizon don't want to charge the same amount for unbalanced peering traffic, as each has different costs for the delivery of that traffic. If L3 can charge X dollars to deliver traffic over optical fibers to routers around the world, and Verizon has to charge Y dollars to deliver that traffic to peoples homes and mobile devices, I can easily understand how X and Y may be different costs, and Verizon needs to charge appropriately based Verizon's delivery costs. If Verizon's cost to deliver traffic to customers is higher than L3 currently charges, there is pricing back pressure for L3 to charge THEIR customers an appropriate amount.

    It could also be L3 has signed contracts to deliver Netflix traffic at a certain cost, and paying Verizon unbalanced peering fees is not in their pricing model. Please clarify the situation L3, I'm not buying your finger pointing at Verizon. Offhand, if its a balanced peering agreement, Verizon is already allowing unbalanced traffic flows, which L3 should already be paying for, so Verizon seems like they are already being very generous to L3.

    Of course we don't know what the details of the peering agreement between L3 and Verizon says, so we can't really point fingers with any accuracy, we can just speculate based on PR spin being done by each party.
    jcbottorff
    • "If one party is sending more traffic, they need to pay the other ISP to de

      "If one party is sending more traffic, they need to pay the other ISP to deliver that traffic to the other ISP to deliver that traffic to it's destination."

      Wrong.

      It is the "other ISP" that is ASKING for the traffic.

      It is not possible to "push" traffic that way. TCP/IP doesn't work that way.
      jessepollard
      • Actually you can

        You can weight IP's to go through certain connections, it is called weighting and you can do it using BGP. heck you can do it on your own computer at home, its called gateway metrics. Usually you do not want to weight the traffic as the traffic is going the most efficient way (fastest) but it can be done and is done.
        schultzycom
    • If they need equal data going each way

      If the need for more data going the other way in order to get them even was the real issue, netflix could easily make their app send a stream of random data back to netflix thus balancing it out. Since they don't do this, I suspect that is not the reason. The peering you alude to was set up between the different backbone operators, so that one backbone oerator wouldn't dump all their data onto another. It was not meant for the ISPs. It would be ridiculous to think that consumers on an ISP would ever have balanced data, especially since most of their contracts specifically preclude running a server.
      normcf
    • Hmm

      Since Verizon is requesting the data why would L3 have to pay for it. It would be up to Verizon to pay.
      schultzycom
    • So, Verizon is not satisfying its customers REQUESTS

      Seems to me the request is coming from Verizon customers - people to whom VZ has promised blazingly fast speeds. Now it seems you cannot get those speeds, or only if you use Redbox Instant or other VZ services.
      Harlon Katz
  • Netflix - Fios customer

    I called Verizon FIOS customer service about the terrible Netflix connection. I asked for a credit for 9 months of Netflix fees because Verizon wasn't providing what I pay for. The amazing thing is that the rep said they had never heard anything about this; after putting me on hold for about 5 minutes she came back and said that it was a Netflix issue and tried to sell me a speed upgrade. Told them if they don't want my $5,000 / year I can go to Cablevision. That was the end of the conversation. WTF happened to the premium FIOS customer service of a couple years ago?

    Anyway - anyone have a practical way of dealing with Verizon? They just don't seem to give 2 sh*ts.
    bobfarm
    • i can download gigabytes of data from msdn with no problem ...

      Including 170gb of conference MP4 videos at 50mbps+ but try to watch a Netflix show and it buffers like crazy and lowers the quality to 200p. It don't take anywhere near 50mb to stream a hd video (5 mbps is all you need for hd video.) and it used to be much better before I upgraded to 50mbps.

      Somebody needs to be jacked up over this crap.
      greywolf7
    • I have read one way...

      And that was to use a VPN to connect to Netflix. That way Verison doesn't see that the traffic is Netflix, thus they don't penalize.
      jessepollard
      • found the reference

        http://www.theregister.co.uk/2014/07/18/want_to_beat_verizons_slow_netflix_get_a_vpn/

        It even provides a reference to a video showing the differences.
        jessepollard
    • $400?

      You're paying $400.00+ a month to Verizon? How's that possible?
      dsf3g
      • $400

        Should have mentioned includes wireless, data, tv and phone.
        bobfarm
  • If there is any doubt left..

    If anyone doubts that the reported speed issues are related to either a financial or strategic objective of your ISP, take this into account.

    I live in Australia, and use Netflix via the wonders of unblock-us. I used Netflix all the time, and cannot remember the last time I saw it have to buffer. Perfect video streaming at the other side of the world.
    gr1f
  • Monopoly

    This is what happens when you have a monopoly ISP that also derives profits from delivering content. Netflix and similar services provide content at about 1/10 the price of your typical cable TV subscription. That's why so many of us have cut the cord in the past few years. But every time someone cancels the TV peon of their bill, your ISP likely loses money, so companies like Verizon and Comcast have an incentive to degrade the Netflix experience.

    What's really needed is for the FTC and FCC to treat Comcast, Verizon, Time Warner, etc. like the monopolies that they are and regulate them as such. In much the same way the FTC threatened to break up Microsoft in the 80's when that company was utilizing is OS monopoly to undermine competitors in the software realm, the FTC should demand real corporate firewalls be set up between the ISP sides of the house and the content provider side.
    dsf3g
    • Monopoly

      Hence the calls for FCC to regulate ISPs as utilities so they can be more effectively regulated like the monopolies they are. In the US Supreme Court decision that invalidated the FCC system of regulating ISPs, the majority provided a roadmap, which all but said "regulate ISPs as utilities, and we will have no problem with your establishment of net neutrality..." Unfortunately,the industry-heavy FCC is desperately looking for any way other than utilities regulation, since that would cause their ISP buddies to scream bloody murder.
      morjoievie