X
Tech

Inside the belly of the benchmark debate

David Berlind says Intel's benchmarks don't add up. George Ou disagrees.
Written by Larry Dignan, Contributor

David Berlind says Intel's benchmarks don't add up. George Ou disagrees. The debate on technology benchmarks--whether it's hardware related like Intel vs. AMD or software--is nearly endless.

This Talkback sums up the state of affairs with benchmarks:

When I look at almost any product I can think of, I would not trust the manufactures claims. Instead, I look for independent reviews. There are a number of sites that do real world usage benchmarks (Toms Hardware) and can provide a lot of information on how hardware performs on tasks that you, George, or I might do. That to me means vastly more than some sideways applied counting contest called a benchmark.

Anyhow, yes I agree, manufactures marketing spin in their claims are outlandish, that is why they should be ignored...

The big problem: Technology buyers don't have much beyond manufacturer claims. Benchmarking standards don't exist. Independent third parties are disappearing as tech trade magazines cut back on labs. Reviews are helpful, but benchmarks are still all over the map.

I'm not sure what the answers are to the benchmarking issue, but the problem needs to be fixed across the entire sector. Here's some required reading as we ponder what exactly that fix should be.

Editorial standards