/>
X
Business

Google shares more of its secret sauce: Protocol buffers

It's a common problem in computer science: how do you get data from one part of your program to another part? What if the two parts were written by different people, at different times, in different languages, on different machines?
Written by Ed Burnette, Contributor on

It's a common problem in computer science: how do you get data from one part of your program to another part? What if the two parts were written by different people, at different times, in different languages, on different machines? Search giant Google has to deal with this issue all time time, only at a bigger scale than most of us. This week they shared the solution they use , a home-grown technique called Protocol Buffers.

Protocol Buffers allow you to define simple data structures in a special definition language, then compile them to produce classes to represent those structures in the language of your choice. These classes come complete with heavily-optimized code to parse and serialize your message in an extremely compact format.

You can think of protocol buffers as kind of a cross between XML and IDL (Interface Definition Languages). Compared to XML, they have several advantages for serializing structured data. From the documentation:

 Protocol buffers:

  • are simpler
  • are 3 to 10 times smaller
  • are 20 to 100 times faster
  • are less ambiguous
  • generate data access classes that are easier to use programmatically

Protocol buffers were written by Kenton Varda, based on an original design by Sanjay Ghemawat, Jeff Dean, and others. The package is used widely inside Google:

Protocol buffers are now Google's lingua franca for data – at time of writing, there are 48,162 different message types defined in the Google code tree across 12,183 .proto files. They're used both in RPC systems and for persistent storage of data in a variety of storage systems.

Version 2.0.0 Beta is available for download now. It supports C++, Java, and Python, but you can easily add support for other languages. All the source code is provided under the Apache license, and there are no restrictions on using this in free or commercial software.

Editorial standards

Related

The 21 best Black Friday deals under $30
Amazon Fire TV Stick 4K

The 21 best Black Friday deals under $30

The 62 best Black Friday deals you can shop at Costco right now
LG 65" Class - QNED80 Series

The 62 best Black Friday deals you can shop at Costco right now

The 52 best Black Friday deals you can still shop on Amazon right now
Image of Amazon Echo Show 8 on a wooden table in front of a person cooking and folding pastry dough.

The 52 best Black Friday deals you can still shop on Amazon right now