Innovation

Google's TensorFlow Serving goes open source for large scale machine learning model creation

The software is designed to make developing machine models for large-scale experiments easier.

Written by Charlie Osborne, Contributing Writer Feb. 17, 2016 at 4:43 a.m. PT

Google has released TensorFlow Serving to the open-source community, a fresh addition to computer learning software for large-scale modeling projects.

On Tuesday, Google Software engineer Noah Fiedel said in a blog post the software is a high-performance serving system for machine learning models, especially for production environments.

TensorFlow is machine learning software used in well-known applications such as Google Photos and Google Translate, as well as Smart Reply in Inbox and the speech recognition in the main Google app.

The capabilities of the software can go also beyond bolt-on features for such applications. According to Google, the technology can be utilized by researchers who need to analyze sets of complex data.

Machine learning software, however, presents a number of challenges. Problems linked to implementing machine learning software include high maintenance costs, data dependency and hidden feedback loops.

Back in November, the Mountain View, California-based firm outlined plans to open-source the TeslaFlow machine learning system, which could be used by developers to improve software which harnesses the power of neural networking.

However, there is another slant to consider -- by making such software open-source, more developers may become interested in refining the burgeoning technology -- which may, in turn, lead to new startups or the emergence of talent in this field, of which companies like Google are likely to be interested in to improve their own product range.

Featured

TensorFlow has been released under an Apache 2.0 license and has been designed for traditional PCs and smartphones. TensorFlow Serving, however, is better equipped for running multiple models at large scales "which change over time based on real-world data," according to Fiedel.

These models could include lifecycle management, algorithm experiments or models developed for GPU resource management.

The software allows users to deploy new models and run experiments on the same server architecture and API, and works best with TensorFlow integration. The technology allows for different pipelines, models and projects to run at the same time as updates become available, as shown below:

TensorFlow Serving, written in C++, has also been released under the Apache 2.0 license and supports Linux operating systems.

"TensorFlow Serving introduces minimal overhead," Google says. "In our benchmarks we recoded ~100,000 queries per second (QPS) per core on a 16 vCPU Intel Xeon E5 2.6 GHz machine, excluding gRPC and the TensorFlow inference processing time."

Interested users can clone the code from GitHub.

Google is far from the only company looking towards machine learning to improve its products. Facebook open-sourced its own brand of machine learning software in November last year, and Microsoft followed in January with the open-source push of the firm's Computational Network Toolkit.