1 of 7Image
The Microsoft Experimentation Platform wants to take decision making from the Hippo (highest paid person's opinion) and take the guesswork out of project planning. The goal of General Manager Ronny Kohavi is to build a system that would map user activities using machine-learning algorithms. Rather than make changes on the run, ask your customers first.
Read Mary Jo Foley's interview with Ronny Kohavi.
Measuring the success of a concept should be simple according to this diagram. Take two versions to test, a control--usually the live version, and the treatment which is usually the new idea. Then you simply collect the data and analyze.
Following is from a white paper from Microsoft Experimentation Platform: Practical Guide to controlled experiments on the Web: Listen to your customers not the Hippo.