OnePlus accused of benchmark cheating -- again

OnePlus 5 review units contains code that allows them to manipulate benchmark tests to achieve performance that the end user will not see while running regular applications, claims the well-respected tech site XDA Developers.

See also: Five Apple products that are headed for the scrapheap

Examination of the device showed that the Android operating system loaded onto review units had been tampered with to produce benchmark results that are not representative of real-world usage.

"Unfortunately, it is almost certain that every single review of the OnePlus 5 that contains a benchmark is using misleading results, as OnePlus provided reviewers a device that cheats on benchmarks," writes Mario Tomás Serrafero on XDA Developers. "This is an inexcusable move, because it is ultimately an attempt to mislead not just customers, but taint the work of reviewers and journalists with misleading data that most are not able to vet or verify. As a result, every OnePlus 5 review citing benchmark scores as an accolade of the phone's success is misleading both writers and readers, and performance analyses based on synthetic benchmarks are invalidated."

The cheat code is designed to manipulate results for the following benchmark tools:

• AnTuTu
• Androbench
• Geekbench 4
• GFXBench
• Quadrant
• Nenamark 2
• Vellamo

The cheat code spoofs the benchmark tool by causing the system to run at a higher than normal CPU speed during the benchmark.

The charts below show CPU frequencies when running GeekBench 4 downloaded from Google's Play Store compared to XDA Developer's "hidden build" of GeekBench.

"In case it isn't evident from the graph above," continues Serrafero, "we polled the CPU frequency every 100ms, and in total, only 24.4 percent of readings returned the maximum frequency of 1.9Ghz when disabling cheating. Meanwhile, the run with enabled cheating spent a staggering 95 percent of readings in its maximum frequency state."

The cheat code boosts the OnePlus 5 multi-core performance by around 5 percent, which doesn't sound much, but as Serrafero points out, "that nudge is enough to propel the device ahead of other Snapdragon 835 devices."

Another undesirable side-effect of forcing the CPU to run so fast is that it causes the device to heat up to the point where it can become uncomfortable to hold. While running the GFXBench benchmark, the outer shell of the OnePlus 5 was recorded at 50 degrees C | 122 degrees F, which Serrafero described as "scorching hot for a phone, and is thoroughly uncomfortable to hold."

OnePlus has responded to the criticism with the following statement:

"People use benchmark apps in order to ascertain the performance of their device, and we want users to see the true performance of the OnePlus 5. Therefore, we have allowed benchmark apps to run in a state similar to daily usage, including the running of resource intensive apps and games. Additionally, when launching apps the OnePlus 5 runs at a similar state in order to increase the speed in which apps open. We are not overclocking the device, rather we are displaying the performance potential of the OnePlus 5."

This statement doesn't make sense, since this CPU performance boost only occurs when selected benchmark tools are run.

This is not the first time that XDA Developers have pointed the finger at OnePlus for benchmark tampering. Back in January, a cheating mechanism was found in the code that shipped with the OnePlus 3T.

12 things Steve Jobs would hate about Apple today

Other stories:

• iOS 11 has a big problem, and we're to blame for it
• The iPhone 8 will have twice as much glass for you to break
• iPhone's battery draining fast? Here's how to fix it
• The quickest, simplest way to speed up an old, tired PC
• Apple selling outdated 32-bit iOS apps that will soon stop working
• How to securely erase hard drives (HDDs) and solid state drives (SSDs)