X
Business

Google's secret home-brew 10 gigE switch

Has Google has built its own 10 Gigabit Ethernet switch - one far cheaper and power-efficient than existing switches - as it seeks to widen its infrastructure cost advantage over Yahoo, MSN and AOL?Andrew Schmitt, an analyst at Nyquist Capital makes the case.
Written by Robin Harris, Contributor

Has Google has built its own 10 Gigabit Ethernet switch - one far cheaper and power-efficient than existing switches - as it seeks to widen its infrastructure cost advantage over Yahoo, MSN and AOL?

Andrew Schmitt, an analyst at Nyquist Capital makes the case.

Through conversations with multiple carrier, equipment, and component industry sources we have confirmed that Google has designed, built, and deployed homebrewed 10GbE switches for providing server interconnect within their data centers.

Lean and mean Why does Google care? They've already made major power and cost cuts in their server infrastructure: low-end PC designs; direct-connect disks velcro'd to caseless mobo's; and cheap, unmanaged fast Ethernet switches tying local nodes together.

After cutting the big numbers it is time to cut the smaller ones. They avoid several power and money hungry features of costly enterprise switches. Twinax cabling for short runs. Ancient 850 nm optical interfaces and cabling for longer runs up to about 100 m. No equalization. Inexpensive switch chips from Broadcom.

Why 10 gigE? The Google File System and Bigtable are designed to use cheap, slow networks. That's why there aren't any NAS or RAID arrays in the main Google infrastructure: too much network overhead. Not to mention the cost.

What Andrew doesn't explore is why Google needs 10 gigE. Because it's there? No. They need the scale out.

Scale-out heaven Let's say you had a warehouse with 300,000 Core Duo 2 servers and a rapidly growing workload. Each query slogs through 70-100 MB of data to find your Phoebe Price photos. That traffic is mostly local, but more is on the way.

Machine translation. Cell phone ads. Hybrid maps. Collaboration apps. Bigger social networks. And more.

The 10 gigE switches mean that dozens of gigE uplinks from cheap unmanaged switches can be aggregated on 10 gigE links, increasing the span of 2nd level switches and eliminating the need for a 3rd switch tier. This reduces power, cost and latency, at not much more cost than standard 2nd tier switches today.

The Storage Bits take Google's focus on power is paying dividends for the entire industry today. Their focus on cheap, high-speed networking will also drive 10 gigE down the price curve faster than Cisco and Juniper would like.

At the same time their infrastructure cost advantage will continue to grow. Bad news if you are trying to compete with Google.

Comments welcome, as always.

Editorial standards