AMD removes bogus benchmarks
Summary: [Update 8/17/2007 - AMD caught passing junked benchmarks red handed] [UPDATE - 1:00 PM - AMD has done the right thing and removed the objectionable slides this morning. Thank you AMD.
[Update 8/17/2007 - AMD caught passing junked benchmarks red handed]
[UPDATE - 1:00 PM - AMD has done the right thing and removed the objectionable slides this morning. Thank you AMD. Blog title was changed from "AMD drags feet on taking down bogus benchmarks".]
There's been a lot of controversy and debate when I exposed AMD for posting deceptive benchmarks on Barcelona and it lead to an announcement from AMD that they would take down the deceptive benchmarks and post honest scores. There's just one problem; AMD intends to take their time taking down the grossly misleading scores. In fact we may not see anything change for a few weeks until newer "honest scores" are posted. After a lengthy back and forth email discussion with AMD yesterday where I tried to reason with AMD that it would be best to remove the deceptive scores immediately, I've been informed that they "will ultimately be down" and that AMD will be sure to alert me when they are down.
In light of AMD's convenient feet dragging, it's hard to believe that AMD is sincere that they were just innocent bystanders.If you're wondering why this even matters, AMD vigorously defended itself by saying that they posted the simulated Barcelona 2.6 GHz scores - a product so far out that it doesn't have a launch date - and older Intel quad-core CPUs back in April. So therefore AMD wasn't aware of the much better Intel scores as if to imply that AMD didn't post deceptive benchmarks at the time it was posted. The problem is that AMD still posted scores for a non-existent AMD product based on "simulated" results which is completely bogus and deceptive.
The bigger problem is that even though AMD posted these deceptive benchmarks back in April, it wasn't widely reported and not many people knew about it. But sometime this week, these deceptive charts were widely distributed among the press which led many technology news sites to blindly regurgitate these deceptive AMD benchmarks as proof of how wonderful AMD's new Barcelona product will be. As an example of how widespread these bogus benchmarks have spread, here's a list of stories glowing about Barcelona citing AMD's simulated Barcelona 2.6 GHz scores going up against older Intel products with lower outdated scores.
- The Inquirer
- TechReport (Parroting the Inquirer)
- x-bit labs
- bit-tech
- TGDaily
So we have a situation where AMD distances itself from the news sites that are regurgitating these bogus numbers that AMD has already agreed to take down. AMD then denies distributing the deceptive slides yet everyone points back to AMD being the ultimate originator of the slides and everyone points back to AMD's website as validation. Now I have no reason to believe AMD is not telling the truth that they didn't send the slides, but the slides on their production website is STILL acting as an official source of validation. So regardless of whether AMD sent the slides or not, AMD intends to conveniently benefit from the false accolades being given to them by these news sites regurgitating AMD's deceptive benchmarks because AMD won't immediately take down the deceptive benchmarks. In light of AMD's convenient feet dragging, it's hard to believe that AMD is sincere that they were just innocent bystanders.
AMD has promised to take down these deceptive benchmarks but they intend to take their time doing it. I've already asked AMD about 5 times to take down those deceptive benchmarks as soon as possible via email and I don't seem to be getting anywhere so I will bring this issue to the public. Hey AMD, take down those deceptive benchmarks now!
Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.
Talkback
Good job sticking to your guns!
In the end, it probably reminded Hector they should behave responsibly and not let AMD marketeers make up for the crummy early performance numbers of Barcelona.
That's a big part of the problem, they think they can get away with this.
As long as AMD believes they can get away with this, they'll keep pulling this nonsense. I think this time they're actually getting burned for a change and they might eventually get the message.
Oh really?
Really? I thought the obligation of the "press" was to INFORM.
Nothing George, on the various bizarre settings in benchmarks (BIOS, prefetch, combining 32-bit and 64-bit compiles, etc.).
Nothing George, on SPECfp comparisons.
Nothing George, on what benchmarks are valid and what they measure, and what benchmarks are useful to customers.
Nothing George, on memory bandwidth, I/O bandwidth, system balance, and what matters for particular applications (stunning since you claim to be an IT consultant). All the more interesting since Oracle continues to recommend Opteron over Woodcrest and Clovertown. I am not sure why, but all of my Oracle RAC customers are running RAC on Opteron, despite what benchmarks say. One would think a journalist would call up Oracle and find out what's up?
Nothing George, on Intel's inability to get Core2 into anything larger than a two-socket machine. Nothing George, on the limitations of a two-socket "quad-core" system versus a four-socket dual-core system (memory capacity, memory bandwidth, I/O slot count, I/O bandwidth, etc.). Nothing on what Clovertown is good for (other than SPECint). This from a claimed IT consultant!
Nothing George, on software licensing costs and price/performance of Clovertown vs. Woodcrest. Clovertown may be cheap per core, but it costs $30,000 more in Oracle licenses per socket for Clovertown vs. Woodcrest. Is it worth it?
Another poster was wrong when he said you were not a journalist. Journalists today are awful (I had a college roomate who was a journalism major, so I know what they were teaching 20 year ago). Journalists today are more interested in their own self promotion, "gotcha" journalism, sensationalism, created scandal, and a "good story" (direct quote of Peter Jennings from an ABC World News Tonight add, where he says journalism is about good story telling, not about recounting the facts).
George, rest assured, you are a journalist. The only thing missing is you need a flash animation on your blob with a "Ou News Alert" ala Fox News.
I am by no means an AMD apologist (I'm writing this from my nice Intel Core2 Duo machine), but I will be watching your blog very closely if Montreal ships in 2008.
Maybe George should write something about...
Maybe George should write about Hector's golden parachute in his employment contract or about the indemnification clause built into the Senior Notes that absolves AMD officers from being sued for running the company into the ground.
I can give George as many ideas as you.
Are you a fired AMD emloyee?
And how Hector's golden parachute affect how good work the engineers done on Barcolona?
Are you a current AMD employee?
Not at all
English is not my native language. You know, Earth is big.
Not at all...oh yeah?
And I don't hate AMD. You are misreading and debating without much credible collateral.
AMD got itself in this mess because if failed to execute. And I am sorry if you can't see through it.
Really?
Okay, it would have been better if AMD were already releasing Barcelona at 3 GHz. Right. But you are downplaying or simply ignoring all the advantages of their architecture and platform, which at least compensates for the not yet most optimal clocks.
You are also seem to object AMD bought ATI. Right? So you don't see the reasoning behind it. Right? Ohh, aren't you a shareholder of AMD instead, who lost some money, or afraid of it?
Really? It's this simple...
Insults? I have not even started.
<b>Where are the reasonable arguments: none that I have seen yet from you</b>
<i>Okay, it would have been better if AMD were already releasing Barcelona at 3 GHz. Right. But you are downplaying or simply ignoring all the advantages of their architecture and platform, which at least compensates for the not yet most optimal clocks.</i>
George's post and susequent replies had nothing to do with architectural differences between Intel and AMD. The issue was all about AMD comparing simulated benchmark numbers (arguably what AMD hoped to come out the chute with at Barcelona launch) vs Intel's 2nd best. They got caught, admitted the comparisons were not correct, and agreed to remove the material.
<i>You are also seem to object AMD bought ATI. Right? So you don't see the reasoning behind it. Right? Ohh, aren't you a shareholder of AMD instead, who lost some money, or afraid of it?</i>
Show me where I onced mentioned AMD's ATI acquisition. The reasoning might have been valid had they closed on the deal six months earlier but unfortunately they bit a bit more than they could chew.
AMD got complacent just like Intel got complacent with netburst. Now they are paying the consequences of trying to become like Intel too soon with market changing conditions.
So please, get off whatever pony you are riding and put your feet on the ground, for once.
It's you
What reasonable arguments? Like that the graphs were made earlier, not after the so called bad news. And that it was accurate at the time (up to date numbers for Xeon, and reasonable numbers for Barcelona, according to the earlier plans). And that it's incorrect to compare more fresh numbers (better optimization by newer compiler version) on Intel's side, to old numbers on AMD's, and say that's the very truth. And so on.
You said: "I'll bet you'd probably object to George writing about AMD having to go into serious debt because of mis-management too, right?" Didn't you referred to the acquisition of ATI?
It's you -?
Is this a newsflash? Or maybe I could spoon feed you multiple sources expressing 2.0Ghz initial Barcelona speeds as un-competitive? But I will not - you can do your own homework.
"What reasonable arguments? Like that the graphs were made earlier, not after the so called bad news. And that it was accurate at the time (up to date numbers for Xeon, and reasonable numbers for Barcelona, according to the earlier plans). And that it's incorrect to compare more fresh numbers (better optimization by newer compiler version) on Intel's side, to old numbers on AMD's, and say that's the very truth."
Sigh...there are no reasonable hard numbers other than powerpoints, the Povray fiasco, and a stealth measurement (cinebench)...but again: you will have to do your own homework.
"You said: "I'll bet you'd probably object to George writing about AMD having to go into serious debt because of mis-management too, right?" Didn't you referred to the acquisition of ATI?"
Silly - AMD's purchase of ATI was cash and stock: the debit I am refering to is the convertible senior notes.
This just goes to show your, er, what was it? Ah...yes 'Technical Ignorance'
QED
Yours
You said: "A profitable AMD is an unreal expectation".
"Sigh...there are no reasonable hard numbers"
I said reasonable arguments. Can you figure the difference?
"other than powerpoints"
What powerpoints?
"the Povray fiasco"
Let me tell you something about POW-Ray Benchmark: if you run it the standard way, and so with the standard settings, there will be no rendering window. Now look at the screenshots more closer: there it is. And there are of different size. Now let me tell you some other thing about POW-Ray: it's all affecting pixel/sec numbers. "reasonamble numbers" LOL!
"and a stealth measurement (cinebench)"
Let me tell you something about Cinebench: what's affect it most is clockrate. Go figure.
"This just goes to show your, er, what was it? Ah...yes 'Technical Ignorance'"
See Subject.
Yours [corrected]
I could tell you some advice what to do with them. Look for a message below with a subjest starting as "Otherways" for my opinion on this.
"Sigh...there are no reasonable hard numbers"
I said reasonable arguments. Can you figure the difference?
"other than powerpoints"
What powerpoints?
"the Povray fiasco"
Let me tell you something about POV-Ray Benchmark: if you run it the standard way, and so with the standard settings, there will be no rendering window. Now look at the screenshots more closer: there it is. And there are of different size. Now let me tell you some other thing about POV-Ray: it's all affecting pixel/sec numbers. "reasonamble numbers" LOL!
"and a stealth measurement (cinebench)"
Let me tell you something about Cinebench: what's affect it most is clockrate. Go figure.
"This just goes to show your, er, what was it? Ah...yes 'Technical Ignorance'"
See Subject.
Yours [corrected]
AMD apologist - end of story
outrageous
They made it worse with all of the Intel downplay and wiseguy antics, and that at times when they were hurting.
What they portrayed as confident agression in front of the IDF in 2003/4 with flyers about 4p server sytems truly showed its colors in recent times as teenage angst and market jealousy.
Otellini was right when he said that AMD failed to plan ahead financially by building fabs to saturate the market ; Moreover they failed to wrestle themselves into the Joe consumers household with adequate advertising.
they arent running it into the ground, they are running it improperly versus the public image persuit that Intel has made very popular with the blue men and sold all the idiots bad procs on more than one occaision.
3-4 guys with blue paint sold billions in chips,,,,,I didnt even know there was an AMD until I bought one from a system builder. I knew there was an intel and i didnt even own a pc yet.
The hush model is broken and AMD needs to advertise in a non hype non downplay posotive manner. They have never been in this position to need to think about advertising and now the transition to needing a new model is brutalizing them.
Out with the old marketing staff and in with the new, they dont need the best chips to sell large numbers, the chips are good enough, they arent prescotts for gods sake.
they are way too attractive in the tech market to not advertise attractively on television.
Intel could...
Intel Marketing folks: if you are reading this, take note.
Yeah
Their manner is to not acknowledge the competition and agressively out manuever. That seems to pay in spades for them.
AMD Will Continue for as long as they "HAVE TO"
AMD cant afford it, they cant just sit around for a couple years letting the truth run rampant that they currently cant compete with Intel CPU's, they are not big enough to bare the brunt of that blow like Intel did for so long. SO until AMD doesn't HAVE TO misdirect the public to keep their heads above water they will continue to to do so. All they are waiting for now is a suitably nonsensical replacement for the deceptive benchmarks they say they will remove.
A very important correction!
[b][i]"...AMD cant afford it, they cant just sit around for a couple years letting the truth run rampant that they currently cant compete with Intel CPU's..."[/i][/b]
This is the problem! That is absolutely FALSE! AMD [u]does[/u] have processors that compete very favorably with [u]every[/u] Intel processor, except at the very top (currently about $1000). In fact, AMD processors are priced such that they offer solidly superior value -- same performance for less money (0).
Few customers buy that fastest $1000 Intel processor, but the "Speed Crown Effect" illogically affects perceptions and buying decisions at all price-performance levels.
[b]The "Speed Crown Effect"[/b]
For a few years, and up until about a year ago, when AMD had the world's fastest X86 processor, many people incorrectly thought "all AMDs are faster than all Intels" and Intel was losing market share each year ("Pentium D is a dog" and all of that...not true). But, it was only the top AMD processor that was faster. This is the "Speed Crown Effect".
Now that Intel has the world's fastest X86 processor, many people incorrectly think "all Intels are faster than all AMDs" and AMD is losing market share. This is really silly. Both companies have always had great products!
[b]The price war is bleeding AMD to death[/b]
AMD is still selling a lot of processors because not everyone is fooled by the "Speed Crown Effect". The problem is, Intel is bleeding AMD to death in a price war at every price point where AMD has competitive products to offer (i.e. everywhere but at the top). Intel is financially much bigger and stronger than AMD. Intel can afford to reduce prices and lose money on processors without going out of business for longer than AMD can. This is a pretty vicious bloody fight.
For example, the fastest dual-core processor that AMD has ever created (Athlon 64 X2 6000+) is now priced at just $169 (0)! About a year ago, when AMD had the "Speed Crown", AMD's fastest sold for about $1000. And, that processor was slower than this one.
[b]What AMD must do[/b]
In order to stay in business, AMD must retake the "Speed Crown" from Intel and resume selling those $1000 processors (even more expensive for servers). AMD will then get the "Speed Crown Effect" and will be able to raise prices and return to profitability.
[b]What all of us must do[/b]
All of us have a stake in AMD. Why? If AMD dies, the fastest Intel processor will cost $1500 and the slowest will cost $800. We all lose if AMD dies.
If you need the speed that you can only get from that $1000 Intel processor, by all means buy it. But if you don't, always buy the AMD processor that meets your performance needs and fits your budget. See (0).
Disclaimer: I am not affiliated in any way with AMD.
----------------------------------------------
(0) AMD and Intel processors with equivalent PC WorldBench scores
[pre]
Processor PC WorldBench Price (7/10/2007)
---------------------------------------------------------------------------
AMD Athlon 64 X2 4600+ (2.40 GHz) 124 (1) $108 (3)
Intel Core 2 Duo E6300 (1.86 GHz) 122 (1) $163 (4)
AMD Athlon 64 X2 5000+ (2.60 GHz) 130 (1) $138 (3)
Intel Core 2 Duo E6400 (2.13 GHz) 131 (1) $183 (4)
AMD Athlon 64 X2 5600+ (2.80 GHz) 138 (1) $148 (3)(5)
Intel Core 2 Duo E6600 (2.40 GHz) 140 (1) $222 (4)
AMD Athlon 64 X2 6000+ (3.00 GHz) 489 (2) $169 (3)
Intel Core 2 Duo E6700 (2.66 GHz) 482 (2) $317 (4)
Note: This Intel CPU fares better on the
Photoshop test (2).
AMD currently has no match for this performance level
Intel Core 2 Extreme X6800 (2.93 GHz) 156 (1) $975 (4)
[/pre]
(1) PC WorldBench 5 tests
http://www.anandtech.com/cpuchipsets/showdoc.aspx?i=2802&p=5
(2) WorldBench 5.0 Office XP SP2 Tests
http://www.hothardware.com/viewarticle.aspx?page=5&articleid=929
(3) AMD Athlon X2 CPU Prices
http://www.zipzoomfly.com/jsp/ProductList.jsp?ThirdCategoryCode=110501&SortBy=A
(4) Intel Core 2 Duo CPU Prices
http://www.zipzoomfly.com/jsp/ProductList.jsp?ThirdCategoryCode=110514&SortBy=A
(5) AMD Athlon 64 X2 5600+ 2.8 GHz is equivalent to the AMD Athlon FX-62 2.8 GHz tested in (1).