Search can't scale without social, and why Bing could still win

Web search remains the heart of the Internet. It's our near-universal homepage. But, increasingly, it fails to deliver results that were as useful as they once were, for two reasons:

1. The shear volume of web data continues to rise at a brutally quick pace
2. SEO is being used to manipulate search results with subpar content to turn a quick buck

Google has been flailing throughout 2011 to try to fix the second issue (see Panda fiasco). As for the data volume issue, Bing director Stefan Weitz insists that there's one big answer for solving it: tapping social.

"The infusion of the social signals into search is the next generation of search," said Weitz.

That won't surprise anyone. In fact, Google finally recognized this in 2011 with the launch of Google+ and its tight integration into Google search. However, while Google views social as a layer that adds value on top of its search, Weitz and Bing view it as an central element of the future of search. And, what most people don't realize is that Bing already has more and better social data than Google because of Bing's strategic partnerships with Facebook and Twitter.

"The infusion of social data is not a gimmick," Weitz said. "Literally, without it, search is not going to scale."

Weitz cited the oft-quoted statistic that throughout all of human history up until 2003 we created 5 exabytes of data (five billion gigabytes). We now create that much every day. In 2011, we'll create 1.8 zettabytes of data (a zettabyte is a 1000 exabytes). That's up from 1.2 zettabytes in 2010, and we'll be creating over 20 times that by 2020.

In trying to scale to meet this data explosion, "our search engines themselves as getting creaky" said Weitz. "We rely on people to do things every day. Today's search simply doesn't take that into account. It's completely mathematical... It's been in the current stagnation for the past 12 years."

While Google has been busy working on building Google+ as its social tool, Microsoft has quietly gone out and cut partnership deals with Facebook and Twitter and started integrating their social data into Bing search results. For example, if you do a search on Bing and you're logged into Facebook in the same browser then the search results will show which of your friends have liked a certain page. See the example below:

"The Facebook integration is pretty cool. It brings in your friends to every query," said Weitz. "We built Bing as this 'decision engine' thing." Right now, the endgame for social on Bing involves taking Twitter and Facebook and "embedding those people into the decision flow," Weitz noted. This is becoming more important as people turn to social networks more often for asking questions, as illustrated in the chart below, which is based on a Bing survey.

Weitz didn't hesitate in stating that Bing is ahead of Google in terms of powering search with social. He said that Bing has better social data and is more deliberate about integrating it. He also said that Bing has a lot more social integration coming soon. Inside Microsoft, the Bing division now has a new headquarters at the company's Bellevue campus, has an international team working on refining search, and has a special team working on social integration right in Google's backyard in Mountain View, California.

Earlier this year, Bing reached 30% market share of all web searches among U.S. users. Will social be the key to unlock more gains against Google?

Sanity check

Bing may have a leg up on Google in social today because of the Microsoft deals with Facebook and Twitter, but you also have to keep in mind that Google is going to have more control over its social-search destiny by building its own product. It won't have to worry about partnership deals going bad or having to ask its social partners for additional API access. Google can just make it happen.

On the other hand, social is all about the people. And while Google+ is off to a great start, Facebook and Twitter still own the social graph today and will for years to come. That gives Bing a strategic advantage for the next stage of web search, for now.

Sidebar: Bing's Twitter and Facebook deals

Here's the timeline of Microsoft's Twitter and Facebook deals involving Bing, based on information from Microsoft:

Oct. 21, 2009: At Web 2.0, Qi Lu, president of Microsoft's Online Services Division, announces a new beta feature that enables people to easily search Twitter's real-time information feed within Bing. The feature helps customers to more fully understand the conversations taking place in Twitter by showing the most popular links people are sharing via Twitter and revealing the comments from the most authoritative users about those links. The feature refines the real-time feed by removing duplicates, links to adult content and tweets containing spam.

Feb. 5, 2010: Bing announces an expansion of its global search alliance with Facebook. As part of the new global agreement, Facebook customers searching for Web content on the site are presented with a fully integrated Bing experience, allowing them to more easily search the Web and make better decisions. Microsoft also no longer represents Facebook display advertising sales in the U.S. as part of its ad network offering.

April 13, 2010: Bing announces the limited release of Twitter integration into Bing search results. More specifically, Bing pulls in social content generated on Twitter to surface the most relevant updates quickly following a breaking news event. Bing analyzes what topics are generating the most interest on Twitter and surfaces the latest and most interesting content. It also utilizes Twitter data to bring customers the most popular shared links for navigational queries.

Oct. 13, 2010: Bing announces a deeper alliance with Facebook, offering a more a personalized search experience for people who use Facebook and Bing. The new features are available when an individual is logged into Facebook while searching on Bing and include Facebook profile search and "Like" results. Bing is the only major search engine that can search public "Like" information and surface it to friends, providing a personalized search experience for each customer.  We intend to build upon this alliance to provide a more personalized and improved search experience in the future.

May 17, 2011: Bing announces an expansion of the integration of "Like" results to include not only pages and links liked by a person's Facebook friends, but also results related to trending topics, articles and Facebook fan pages. Facebook Profile Search capabilities are enhanced to return more detailed information when searching for a specific person on Bing. Additionally, social results are integrated into Bing travel and Bing shopping, allowing people to get help from their trusted friends when making decisions online.

Sept. 6, 2011: Bing and Twitter announce they will extend their alliance, allowing Twitter's fire-hose to feed into the Bing real-time search feature.