Benchmark vendor comes down on Samsung, Sony phones for gaming tests

Summary: Last year, a number of Samsung phones were found to be falsely inflating Primate Labs' popular Geekbench 3 benchmarking tests. The software maker recently reported the results of its benchmark boosting detection and how it will deal with the gaming of its system.

In a post to the Primate Labs blog, company founder John Poole laid out the issue of phone makers' recent gaming of its benchmarking tests and the results of its in-house investigation.

Benchmark vendor comes down on Samsung, Sony phones for gaming its tests

Last year, several tech sites discovered that Samsung was fudging its test results with speed-boosting code . The artificial results inflated a few tests by as much as 20 percent, Poole reported.

Check Out: Samsung caught fudging benchmarks (again)

Poole said the company added a "boost detector" into Geekbench 3, which embedded a special report into each test result uploaded to the Geekbench database. Here's the company's list of gamed phones:

Samsung Galaxy Note 10.1 (2014)

Samsung Galaxy Note 2

Samsung Galaxy Note 3

Samsung Galaxy S 3

Samsung Galaxy S 4

Sony Xperia Z

Sony Xperia Z Tablet

Sony Xperia Z Ultra

Sony Xperia Z1

Sony Xperia ZL

The speed boost was found in Android 4.3 only, he wrote. Unlike the asterisks found in the Major League Record Books, Primate Labs pulled the problem results.

In order to combat benchmark boosting we have decided to exclude results from these devices running Android 4.3 from the Android benchmark chart. This way the results on the chart reflect the true performance, not the boosted performance, of each device. We have also added a list of excluded devices to the chart. We will continue to monitor the detector reports, and we will update this list if we discover other devices or Android versions that apply a benchmark boost.

There is one bit of good news that our detector uncovered — Samsung removed the benchmark boost from their Android 4.4 update. We hope that Sony follows Samsung's lead and also removes their benchmark boost from their Android 4.4 update as well.

Dudes, what were you thinking?

Topics: Samsung, Android, Smartphones

About

David Morgenstern has covered the Mac market and other technology segments for 20 years. In the recent past, he founded Ziff-Davis' Storage Supersite, served as news editor for Ziff Davis Internet and held several executive editorial positions at eWEEK. In the 1990s, David was editor of Ziff Davis' award-winning MacWEEK news publication a... Full Bio

zdnet_core.socialButton.googleLabel Contact Disclosure

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Related Stories

The best of ZDNet, delivered

You have been successfully signed up. To sign up for more newsletters or to manage your account, visit the Newsletter Subscription Center.
Subscription failed.