Siri: Will Microsoft's Bing give me quality search results?

Siri: Will Microsoft's Bing give me quality search results?

Summary: The Bing research team is continuing to blog about how it has set about to improve search quality, including user satisfaction with search results.


When Apple officials announced in passing this week that Siri will use Bing instead of Google as its default Web search engine with iOS7, I saw more than a few skeptics wondering aloud about how the Bing results would compare, quality-wise.

I don't have any inside information as to how/why this collaboration came to pass, but it does seem Apple is intent on finding ways to reduce its dependence on Google, even if it requires teaming with another "enemy." Did comparative quality of search results have any bearing on the decision at all? Again, I have no idea, but I'm doubtful.

Just because I tend to believe politics trumps the "best" solution doesn't mean I am down on the quality of Bing's search results these days. As I've noted before on this blog, I've been using Bing with good success on my PCs, tablets and laptops in the past several months; on my phone, I haven't been as happy with the quality of my local Bing search results.

Microsoft officials are confident they can match, if not surpass, other search engines in terms of the amorphous concept of "search quality." In March 2012, Microsoft launched a "Bing Search Quality Insights" series of blog posts to explain what the company has been doing to improve overall search quality in Bing.


The latest installment in that series, today's June 12 post on experimentation and continuous improvement, focuses on techniques that the research team in Bing is using to improve the quality of search results. The post highlights a technique called "optimized interleaving," which combines result lists and tracking clicks to improve the relative quality of search results. The post highlights the findings in a research paper on measuring search results which was presented at the WSDM-2013 conference in February 2013.

Bing's research team, which includes a "few dozen" people, has been working on search for about a decade, said Bing Corporate Vice President of R&D Harry Shum. It's taken much of that time for Microsoft to improve its search quality so that it can stand against anyone else's, Shum said. 

Exactly how one measures search quality remains a thorny problem, Shum said. Just because users get a ranked list of results doesn't insure that those at the top are definitely the most relevant.

The machine-learning systems powering Bing's back end already can handle tasks ranging from correcting spelling of queries, interpreting search intent, separating out junk pages and rank indexed pages. But these kinds of systems need to be optimized for user satisfaction, Shum noted in today's blog post. But this remains problematic because there's no objective way to measure user satisfaction, which is where "surrogate measures," such as the aforementioned interleaving algorithm fits in.

"Surprisingly, once your machine learning systems are powerful enough, your choice of surrogate measure has a strong influence on the kind of results you return," Shum blogged.

Bing researchers used a variant of A/B testing -- randomized experiments with two variants, often used in the medical field -- to create a solution that allows Microsoft to move search results that users prefer higher up in rankings. (The "interleaving" comes into play by allowing users to mix A results and B results, rather than simply choosing one set over the other.)

Microsoft already uses these techniques with Bing today, meaning users regularly take part in A/B experiments when they use Microsoft's search engine, Shum noted in the blog post.

Microsoft as a whole often uses A/B tests to hone products and services, but because Bing has millions of users and billions of clicks, results from these tests are far more statistically significant than they may be in other product groups, Shum said.

More specifics about how the interleaving algorithm works and how Microsoft is using it to hone Bing's results are in today's post. 

Topics: Google, Big Data, Microsoft


Mary Jo has covered the tech industry for 30 years for a variety of publications and Web sites, and is a frequent guest on radio, TV and podcasts, speaking about all things Microsoft-related. She is the author of Microsoft 2.0: How Microsoft plans to stay relevant in the post-Gates era (John Wiley & Sons, 2008).

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.


Log in or register to join the discussion
  • Despite the impression you'd get from the forums here

    I don't think either company regards the other as the enemy. Microsoft once came to Apple's rescue and makes software for Mac. Apple has collaborated with Microsoft on typography, and licensed patents to Microsoft that nobody else has access to.

    So it is no surprise Apple picked Bing. It currently gets on with Microsoft better than either does with Google.
    • Meh

      Depends on the situation. For marketing purposes, your competitors are your enemies. For progression and innovation purposes, your competition can be as useful of an ally as you want. Given Apple is brilliant with their marketing, it isn't surprising that this isn't readily obvious. A company with Microsoft's reach, Apple's marketing, and Google's willingness to create niche programs (as in all the stuff in Google Labs), you'd have an amazing company.
    • Jobs

      Steve made it clear he did not like Microsoft. Sometimes I don't think Steve got it. He once claimed that Microsoft couldn't even copy the Mac -- but what he didn't realize was that Microsoft wasn't trying to copy. Steve, in his ego, just wanted them.
      • One of his many problems

        steve jobs had absolutely no taste. And I don't mean that in a small way, I mean that in a big way, in the sense that he didn't think of original ideas, and he didn't bring much culture into his products.
    • "Microsoft once came to Apple's rescue . . ."

      And yet again the re-telling of a myth. Please do some research as that is NOT what happened!
    • As someone else put it...For the LAST Time!!

      Microsoft did NOT come to the 'aid' of Apple. They took the cheap way out and settled a lawsuit from Apple...hence their paltry 'investment' (which was their best money maker in 10 years) and agreement to provide Office for Mac.

      If anything, Apple did Microsoft a favor by not continuing the litigation and giving MS butt loads of money from their sales and investment revenue.
      Never Use Microsoft Warez
    • Sure. No strife. They're all cuddly.

      This "no strife" message is part of a Microsoft astroturfing campaign you will see all over the internet now - trying to give the impression that these two giants are not fighting. The reason is obvious: having declared total war Microsoft is losing.

      Microsoft is still bashing Google in the press with its "Scroogled" campaign. It still tries to cut off Google's air supply in every place it can. It still has outstanding legal campaigns to have Google's products banned on every continent - directly, through "grassroots consumer groups", "industry trade groups" that it controls, and through puppet patent trolls alone, in partnership with partners like Nokia and less directly as well.

      Steve Ballmer swore to kill Google in 2003. He's trying his best, but getting nowhere. So we have to have folks come here to pretend there is no spat.
  • ios 7 took everything else from Windows Phone

    Why not take its search results too?
    • Everything else?

      Hyperbole much?
      Michael Alan Goff
      • the new modern UI

        Definitely follows MS vision, but it how OSes will all be soon so lets not fall into the "who copied who" it's only phones, but lets just all admit that Microsoft was right. Modern is the way to go.
        Emmanuel Fransson
        • No

          The UI is still rows of static icons.

          The Multitasking is... WebOS.

          Sure, it's flat. But if you think that Microsoft was the first to invent a flat UI then you're wrong.
          Michael Alan Goff
          • again... who was first too...

            Who cares? Its all about timing. I dont care if flat designs were around us for a long time... Its a win when a competitor partly adopts your style (Modern). Now, does it matter? I don't think so.
            Emmanuel Fransson
          • Modern UI

            is a lot more than just flat.

            It's also a lot better than just flat.
            Michael Alan Goff
          • Metro Sucks

            A Static row of animated icons uses more battery power, for stupid things.
            Troll Hunter J
          • Microsoft was the first to popularize it

            And that's what counts, right?

            Xbox and 100 million Windows 8 sales is popular.
          • Popularizing it doesn't mean much

            I don't say Apple invented the smartphone when they were the first to "popularize it". Or how about the iPod? Nope, they didn't really invent anything there either (despite making it way more popular to have a device like that).
            Michael Alan Goff
          • T-Shirts are nothing new

            Yet some companies lead the trends. Just don't be all negative. Its trends... Nothing more and Microsoft was ahead of it. The have lead the trend. Nothing more, nothing less. It happens all the time and there's a new logo associated with the new trend. Now with it be Apple's logo? Maybe, but some of us know where it started.
            Emmanuel Fransson
          • i should read my comments

            Before posting them haha. A lot of autocorrect there hehe. I guess being Swedish doesn't help... I make big mistakes and my phone corrects them wrongly. Have fun reading them.
            Emmanuel Fransson
          • You must be using WinPho8 (NT)

            Never Use Microsoft Warez
    • Congratulations

      You're still the most useless denizen of zdnet. Not sure what the prize for that is, but whatever it is, you're welcome to it.