How Microsoft aims to bring Bing deeper into Windows Blue, Xbox One

In all the talk about Microsoft's makeover into a devices and services company, one service many forget the company has in its back pocket is Bing.

Bing is evolving into more than "just" a Web search engine for Microsoft. It also gives the company a way to harness and make use of data using Microsoft's myriad machine learning and computation capabilities. And though it may be hard to see through all the "Scroogled" fog, Bing may be more important as a service than a "mere" search engine to Microsoft going forward.

This week, as part of its Xbox One reveal, Microsoft execs didn't call out Bing by name much, if at all. However, as a subsequent post on the Microsoft Offical Blog noted, it's Bing that provides the responses when users search by voice via Kinect for movies, TV shows and music. It's Bing that's parsing the natural-language-query commands, such as "Xbox, Snap Internet Explorer." Specifically, it's the Tellme voice technology, combined with social-graph information, plus Bing's search functionality.

A quick Tellme refresher: Microsoft bought Tellme Networks in 2007 for between $800 million and $1 billion.  Tellme provided both a "speech cloud service" and an interactive speech self-service platform that provided interactive voice response, or IVR. (An example of an IVR system is the system that provides an automated voice response when users check on their flight statuses.) Microsoft offloaded the IVR assets to 24/7 in 2012. But it kept the cloud-speech service, which it combined with other internal speech technologies. The cloud service part from Tellme is what is used in Windows Phone, the Bing mobile app, automotive entertainment systems and Xbox Kinect sensors.

Additionally, Bing did work with certain Microsoft Xbox partners, like Netflix and HBO to index their catalogs, alongside Xbox's own game catalog, so that users could search for "Great Gatsby" and see any movies, games, music or other content available through the Xbox. Bing provided the back-end search/recommendation service, starting with the Xbox 360. In a similar way, Bing indexed the Windows Phone app store to provide users with recommendations, in addition to the basic Web search it also provided.

Microsoft is promising the voice-search capability it provides with Xbox and Kinect will be significantly enhanced with the Xbox One. But that's not the only place where Bing is supposed to bring the bling.

Windows Blue, a k a Windows 8.1, has been rumored to include significant Bing improvements. In the leaked Blue builds so far, these search enhancements can't really be seen and tried. But Microsoft's Online Services Division, the unit that includes Bing and the remaining Microsoft Tellme team, has been working with Windows to build a search service that will work across devices, apps and the Web, according to my contacts.

On the apps front, the AppEx team inside Bing is continuing to develop more new Windows 8 apps that Microsoft is expected to roll out when Windows Blue is available this fall. This is the team that built the Weather, News, Sports, Travel and other apps that were preinstalled with Windows 8 and Windows RT. Officials with the AppEx team have said they're working on more, similar kinds of apps. Some new apps -- an Alarms and a Sound Recorder app, specifically -- have leaked as part of the Windows Blue leaks, but I am hearing there will be more. (Update: Supposedly these two already leaked apps were not built by the Bing AppEx team, but were built by a team in Windows.)

The Bing team also is going to power the app-store search and recommendation engine that is part of Windows Blue, I'm told. But Bing is also providing the core search technology for Windows Blue, too, that will improve the search discoverability and relevance in Windows 8.

Instead of having to hunt within Apps, Settings, Mail and other subcategories, users will be able to just start typing and have the operating system figure out for what they're most likely to be searching, one of my contacts said. If that comes to pass, that would be a major improvement over how search currently works in Windows 8 and Windows RT. 

I've been asking around as to whether the Bing/OSD team might be doing anything to improve voice search with Windows Blue, given what they've been doing on the Xbox side of the house. I am hearing from my contacts the answer is no. The reason? Voice isn't so far a priority on PCs/tablets, so Windows Phone and Xbox/Kinect are where the voice focus is at the moment.

That said, Microsoft officials announced on May 23 that a new Kinect for Windows sensor is coming in 2014 and will include improved voice capabilities. So maybe that will include more Bing/OSD capabilities, too.