Images: Testing Microsoft's Experimental Platform

4 of 7
  • 197613.jpg

    The Microsoft Experimentation Platform wants to take decision making from the Hippo (highest paid person's opinion) and take the guesswork out of project planning. The goal of General Manager Ronny Kohavi is to build a system that would map user activities using machine-learning algorithms. Rather than make changes on the run, ask your customers first.

    Read Mary Jo Foley's interview with Ronny Kohavi.

    Caption by: Andy Smith

  • 197614.jpg

    Measuring the success of a concept should be simple according to this diagram. Take two versions to test, a control--usually the live version, and the treatment which is usually the new idea. Then you simply collect the data and analyze.

    Following is from a white paper from Microsoft Experimentation Platform: Practical Guide to controlled experiments on the Web: Listen to your customers not the Hippo.

    Caption by: Andy Smith

  • 197616.jpg

    Ads were palced at the bottom of this shopping section to determine their worth. Clickthrough rate was 0.49 percent lower and page views were down 0.35 percent on the page with extra ads. Apparently not significant but the expected revenue was less than the loss of revenue from lost clicks, so the project was scrapped.

    Click on the image to enlarge.

    Caption by: Andy Smith

  • 197615.jpg

    This experiment ran in Windows Marketplace/Game Downloads. The Solitaire game had a higher clickthrough--61 percent preferred Solitaire.

    Caption by: Andy Smith

  • 197619.jpg

    High-contrast text was tested against softer colors. The high-contrast page drew 0.9 percent more queries per user and 3.1 percent more ad clicks.

    Caption by: Andy Smith

  • 197617.jpg

    Feeback A puts everything together while Feedback B is a two-step process--a question following the ratings. Feedback B got more than double the response rate.

    Caption by: Andy Smith

  • 197618.jpg

    This feedback is also a two-stage process. It outperformed Feedback B by 3.5 times.

    Caption by: Andy Smith

4 of 7

This experiment ran in Windows Marketplace/Game Downloads. The Solitaire game had a higher clickthrough--61 percent preferred Solitaire.

Caption by: Andy Smith

4 of 7

Related Topics:

Enterprise Software Open Source Mobile OS

Related Galleries

    • 1 of 3