AMD to remove deceptive Barcelona benchmarks on website

[Update 8/17/2007 - AMD caught passing junked benchmarks red handed]  

[UPDATE 7/6/2007 - AMD intends to drag their feet on removing the deceptive benchmarks]

A day after I blasted AMD for posting deceptive benchmarks on Barcelona, AMD is promising to release "honest Barcelona benchmarks".  As of Thursday afternoon 7/5/2007 the misleading Barcelona numbers are still posted on AMD's website but AMD is promising to release updated numbers in the coming weeks with Barcelona 2.0 GHz scores (slated for September 2007 release) instead of "simulated Barcelona 2.6 GHz scores".

AMD argued that the charts were created by AMD much earlier in the year, but that's really a terrible excuse since they're now heavily promoting the misleading information.  The charts might have been made earlier in the year but they were "CONFIDENTIAL" and only shown to a few people but Kristopher Kubicki of DailyTech has brought it to my attention that these charts are just now making the rounds in the press and blogsphere courtesy of The Inquirer's Theo Valich drinking the AMD Kool-Aid .  The Tech Report then cited Theo Valich's story as fact.

Valich wrote: For starters, SPECint_rate2006 and SPECfp_rate2006 speak the voice of doom for the 366 MHz faster clocked Xeon - and these are the reasons why Intel was downplaying performance of FP and praising the INT. These figures were from a few months ago, so things may well have changed since then. But in the Integer test, a Barcelona 2.3GHz yields 21% higher score than Clovertown 2.66 GHz, but Floating Point test leaves a staggering 50% performance deficit for Clovertown, and this is not something 45 nanometre Penryn can solve overnight. Unless, of course the clock deficit for AMD is such that Intel speeds past.

Valich made multiple errors in his story.

• First of all he's talking about simulated Barcelona 2.6 GHz scores but he's making it sound like an actual Barcelona 2.3 GHz chip.  Neither the 2.3 nor 2.6 GHz Barcelona chip exists and a 2.0 GHz Barcelona won't ship until September 2007.
• Second, he's citing AMD's outdated numbers and products from Intel.  Intel's 2.66 GHz Clovertown product based on the latest SPEC.org published scores is only 1% slower than the non-existent simulated 2.6 GHz Barcelona scores.  It also doesn't account for the fact that Intel has been shipping a 3.0 GHz Clovertown chip since April which has higher scores than the simulated Barcelona.

The reality is that AMD's claim of a 20% clock-for-clock advantage is completely bogus.  In fact, Nathan Brookwood (analyst at Insight 64) made a clock-for-clock comparison of Intel Clovertown versus AMD Barcelona at 2.0 GHz and found a 4% Integer performance advantage in favor of Intel.  Gartner also cast doubts on Barcelona's claims.

X-bit labs also posted AMD's simulated Barcelona scores on 7/2/2007 and claimed that the Clovertown 3.0 Clovertown wouldn't be released until September 2007.  As Kristopher Kubicki pointed out, it's simply unheard of to cite "simulated benchmarks" in the Microprocessor industry.  Kubicki writes:

So what's going on here?  Shoddy journalism and even the author wouldn't deny that. Any author willing to pad his byline with such omissions of fact would almost certainly have an alternate agenda brewing.

I didn't give these benchmarks much thought when I first saw them in February, and given the confirmed top-out frequency from AMD, I certainly don't give these benchmarks traction now either.  I suggest those with interests in Barcelona wait until someone tests actual DVT or Retail silicon.

The funny thing is I cannot recall the term "simulated benchmarks" in my pre-Barcelona lexicon.  Since when did benchmarking a 1.6 GHz processor give the "simulated" performance of a 2.3 and 2.6 GHz chip?  Does anyone find it odd that Googling for "simulated benchmarks" yields only 574 entries -- all from AMD, Microsoft and Sanda.  AMD holds the number one spot.

Maybe simulated benchmarks will go down with some of the other great marketing terms of the last decade: FUD, paper launches and ship dates.

So there you have it.  AMD might have some plausible deniability here that they officially created these charts back in February, but they have this information posted prominently on their Barcelona product webpage and this information is only now making the rounds in the press as some miraculous triumph at a time when reality is a late underwhelming Barcelona product.  Shame on AMD, I'll be waiting for their "honest Barcelona numbers" to be updated on their product page.