AMD on the attack in processor performance row

Chipmaker AMD has levelled criticisms against a widely used benchmarking suite, calling it biased in the favour of its arch-enemy Intel.

The move steps up AMD's efforts to ensure that its processors compare favourably with Intel's in the public eye, but some observers say both companies' rhetoric should be taken with a large pinch of salt.

In a presentation circulated to several hardware-oriented Web sites, AMD alleges that a recently released version of the Sysmark benchmarking tool, from industry consortium BAPCo, has been revised to downplay the strengths of AMD's Athlon XP processors, while emphasising tests that portray Intel's Pentium 4 in a favourable light. AMD also confirmed that it joined BAPCo earlier this year, with the aim of influencing the way next year's Sysmark is formulated.

Benchmarks create statistics that are supposed to make it easier to compare one processor's performance against another. Sysmark figures indicate a chip's performance in running particular applications, such as office suites, 3D rendering tools and Web applications. Since last year, AMD has been on a renewed crusade to convince users to pay attention to performance measures such as benchmarks, which are supposedly more accurate than clock speed. Athlon chips generally sport lower clock speeds than Pentiums, even if they offer the same real performance.

In its presentation, AMD claims that the way tasks within Sysmark 2002 have been chosen clearly favours Intel's chip architecture. Fourteen tasks that most favoured AMD chips were taken out, AMD claims. "Those accounted for most of our advantage over the Pentium 4," an AMD spokesman said.

The most dramatic change was the way scores for Internet content creation were calculated, AMD said. Compared with scores from Sysmark 2001, Intel chips scored slightly higher, while the score of AMD chips plummeted 20 percent.

An AMD spokesman said that the company had joined BAPCo (or the Business Applications Performance Corporation) in order to have "a more direct interest" in the way the software was formulated. "We want to look at the reasons behind what they did when discussing Sysmark 2003, and have our say as well," he said. BAPCo was formed in 1995 by Intel, Compaq, Dell, IBM and several trade publishers. CNET Networks, the publisher of ZDNet UK, is a member of BAPCo.

BAPCo, for its part, says that it ensures objectivity by building its benchmarks around retail software and focusing on actual PC users' level of usage. Each participating company receives one vote on the benchmark once it is developed.

Intel said that other benchmarks besides Sysmark have confirmed the Pentium 4's performance. "The vast majority have said that the Pentium 4 2.8GHz is the highest-performing processor," said an Intel spokesman.

Some observers have suggested that AMD may be exaggerating its portrayal of the changes to Sysmark 2002. Thomas Pabst, creator of the influential Tom's Hardware Guide Web site, said this week that laboratory tests did not show conclusively that the benchmarks had been changed to favour Intel.

Some industry analysts say that the argument is practically academic, because there is no universally agreed-upon standard for measuring real-world performance. "Benchmark standards are supposed to be objective, so it shouldn't matter who's in bed with whom. It should be irrelevant, but it's not," said analyst Andy Brown of IDC.

Benchmarks could be more convincing if all of the important parties involved agreed to a set of rules, Brown said. "If there were independent benchmarks, conforming to certain criteria, I think perhaps it would be a bit easier for the industry to plead its case," he said.

To find out more about the computers and hardware that these chips are being used in, see ZDNet UK's Hardware News Section.

Have your say instantly, and see what others have said. Go to the Chips Central Forum.

Let the Chips Central editor know what you think by email. And sign up for the weekly Chips Central newsletter.