Benchmark vendor comes down on Samsung, Sony phones for gaming tests

In a post to the Primate Labs blog, company founder John Poole laid out the issue of phone makers' recent gaming of its benchmarking tests and the results of its in-house investigation.

Last year, several tech sites discovered that Samsung was fudging its test results with speed-boosting code. The artificial results inflated a few tests by as much as 20 percent, Poole reported.

Check Out: Samsung caught fudging benchmarks (again)

Poole said the company added a "boost detector" into Geekbench 3, which embedded a special report into each test result uploaded to the Geekbench database. Here's the company's list of gamed phones:

Samsung Galaxy Note 10.1 (2014)

Samsung Galaxy Note 2

Samsung Galaxy Note 3

Samsung Galaxy S 3

Samsung Galaxy S 4

Sony Xperia Z

Sony Xperia Z Tablet

Sony Xperia Z Ultra

Sony Xperia Z1

Sony Xperia ZL

The speed boost was found in Android 4.3 only, he wrote. Unlike the asterisks found in the Major League Record Books, Primate Labs pulled the problem results.

In order to combat benchmark boosting we have decided to exclude results from these devices running Android 4.3 from the Android benchmark chart. This way the results on the chart reflect the true performance, not the boosted performance, of each device. We have also added a list of excluded devices to the chart. We will continue to monitor the detector reports, and we will update this list if we discover other devices or Android versions that apply a benchmark boost.

There is one bit of good news that our detector uncovered — Samsung removed the benchmark boost from their Android 4.4 update. We hope that Sony follows Samsung's lead and also removes their benchmark boost from their Android 4.4 update as well.

Dudes, what were you thinking?