Strike 3 for AMD hypocrisy on benchmarking

Strike 3 for AMD hypocrisy on benchmarking

Summary: AMD swung hard at Intel for what it deemed as ethics violations, but since then, it has pulled the same stunt it's been criticizing Intel for three times in a row. I called out AMD's hypocrisy on strike one; David Berlind called it out on strike two; and now AMD is doing the same thing by cherry-picking the benchmarks and omitting superior Intel scores.

SHARE:
TOPICS: Intel
74

AMD swung hard at Intel for what it deemed as ethics violations, but since then, it has pulled the same stunt it's been criticizing Intel for three times in a row. I called out AMD's hypocrisy on strike one; David Berlind called it out on strike two; and now AMD is doing the same thing by cherry-picking the benchmarks and omitting superior Intel scores. Intel's Nick Knuppffer was rather upset with these omissions today and called AMD's benchmark postings "rubbish."

AMD, in this latest round, showed three sets of benchmarks comparing three products. It shows the latest AMD Opteron 2222 SE, which was released only this month, going against an Intel 5160 from mid-2006 and against an Intel 5355 released November 2006. The problem is that AMD omits the scores on two of the benchmarks and shows only the Intel 5355 in its worst light with SPECompMbase2001, which is a relatively obscure and less often cited benchmark to begin with. AMD selectively omits Intel's 5355 numbers for the more common SPECint_rate2006 and SPECfp_rate2006 benchmarks and then claims victory on all three sets of benchmarks. Here is the chart reconstructed with the best available scores posted by AMD, but I've also included the omitted 5355 scores highlighted in yellow.

Published scores on SPEC.org
 AMD 2222 SEIntel 5355Intel 5160
SPECompMbase2001132751182210689
SPECint_rate2006 Peak56.684.855.2
SPECfp_rate2006 Peak52.160.245.1

Using dual-core 5160 as baseline performance
 AMD 2222 SEIntel 5355Intel 5160
SPECompMbase200124.2%10.6%0%
SPECint_rate2006 Peak2.54%53.6%0%
SPECfp_rate2006 Peak15.5%33.5%0%

So even without Intel's latest CPU in the lineup, Intel wins the two more common benchmarks by a significant margin. The biggest margin of victory goes to the integer performance of Intel's 2.66 GHz XEON 5355, and that gap will get even wider once Intel's just-released 3 GHz quad-core XEON gets included in this lineup.  Earlier today, I explained the benefits of quad-core computing and what lies ahead for the two chip companies.

Cherry-picking benchmarks to show one's own product in its best light is fairly common in the industry, but not after publicly slamming one's opponent for unethical benchmarking. If AMD insists on calling Intel unethical for cherry-picking benchmarks, reasonable people have no choice but to call AMD hypocritical when it does the same thing. Since this is strike three on AMD hypocrisy in barely over a month, this appears to be a pattern that won't stop anytime soon.

Topic: Intel

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

74 comments
Log in or register to join the discussion
  • Which is blacker, the pot or the kettle?

    Don't know, don't care...
    No_Ax_to_Grind
    • Jah we kinda figured

      based on most of your posts.

      :o)
      Jack-Booted EULA
      • So tell me how this effects you

        Do you have a few million $ in stock of one or the other? Are you so limited you actualy either's charts? Do you honestly think being a wee bit less/more guilty make any difference?

        Come on, lets hear how this is a life changing situation for you.
        No_Ax_to_Grind
  • AMD is struggling

    This is a hot topic for sure but AMD takes the cake.
    What we have seen AMD do in the last year is this dismiss benchmarks altogether as a relevant because they are losing them all! Now, they won one that serves as a cherry picked example of superiority.

    The second thing they do is deflect their now uncompetitive microprocessor line in favor of drawing attention to their graphics products. Basically, leverage the equity of the ATI acquisition.
    Prognosticator
  • They didn't publish the best AMD result either

    Since SPEC_IntRate has to do with overall throughput then maybe they went for a closer apples to apples comparison. Looking at the results publishing page (http://www.spec.org/cpu2006/results/rint2006.html)

    You've got an Opteron 8220 entry with 8 Cores on 4 Chips with a Peak of 97.6.

    Heck, go further down the list and you can get an Intel entry at 1590. (HP Integrity Superdome (1.6GHz/18MB Dual-Core Intel Itanium 2 - 128 runs on 64 dual-core CPU's).

    In the case of this benchmark you'd have to at the very least control for the number of base copies (Intel's 80.5 was with 8 base copies while AMD's 51.7 was with 4).

    So, I'd back down on the screaming "hypocrisy" and just acknowledge that this isn't all that great a benchmark for comparing core vs core.
    Robert Crocker
    • Sorry, this is limited to a 2 socket configuration

      Sorry, this is limited to a 2 socket configuration. We're not talking about core versus core, we're talking socket versus socket since that's what counts for power usage, cost of hardware (4 socket is double the server cost), and licensing costs (mostly socket-based).
      georgeou
      • No, *YOU'VE* limited it to a 2 socket configuration

        When AMD was out first with dual core CPU's and the x86-64 instruction set were you so darned "scrupulous" about your demands for equality.

        AMD set up a comparison of 2 socket, 2 core chips and they come out well in that comparison. They were upfront in what they compared, you're just busy griping that they didn't go against Intel's "Quad" (really it's 2x2 and not quad) core offering on a benchmark that is an overall throughput one that is strongly affected by the number of cores or CPU's involved.
        Robert Crocker
  • Can't trust benchmarks from Intel or AMD

    Consumers should never trust benchmarks published by any manufacturer. Every major manufacturer of computer equipment will always cook the results of their benchmark tests to paint their product in the best light.

    If you want to know the truth, you'd better run your own benchmarks or at least look at third party tests that you can find on numerous web sites.
    WiredGuy
    • No Choice

      "Consumers Should".... consumers really don't have a processor choice except that based on total system price. It is difficult to get to the bottom of which processor is "better"... There are so many other factors masking cpu choice.... memory, size hard disk, graphics card... please name a few of your "numerous web sites"

      I just last month bought a Intel Centrino Duo T5500 and tried to do this type of research and failed... especially in the area of laptops. Can the average consumer really understand the meaning of many of these benchmarks? With my EE degree, I should be able to do better.
      wjkahlssmd@...
    • What works for you

      If you work with a computer for a while, you'll lear its strengths and weaknesses.
      There used to be a myth that Macs were slower than PCs when G4s and G5s were
      running at lower clock speeds than Pentium 4s and Athlons. There have been
      times when there was a speed gap, but other times like when G5s came out there
      was nothing in the PC world that would even compare. In fact G5s still out perform
      core 2 duos in some tests, although since Apple uses Intel, you won't hear Steve
      Jobs say that. Also you have to consider that this year's or even this month's
      machine will out perform earlier machines so out and out break-neck speed is
      irrelevant in most cases. All tests are done to favor the product of the person
      paying for the tests.

      The truth is that for most purposes a core2 duo keeps pretty close to an 2x4core
      Xeon, especially for general computing. Still we all want a $10,000 piece of
      hardware to run the $2500 Adobe CS3, but we probably don't actually really need
      either.
      MacGeek2121
    • I said they all pull this kind of stuff

      I said they all pull this kind of stuff. The point is that AMD is screaming out UNETHICAL in front of our cameras and calling out Intel. Now they've done the exact same thing they were complaining about 3 times in a row. That's the issue here.
      georgeou
  • so what already.

    I guess I'm not geeky enough to care... why would I want to read this same story 3 times in a row?
    They're manufacturer sponsored comparisons. Is anybody dumb enough not to take this marketing with a grain of salt?

    Intelligent techs rely on 3rd party tests, not the ones run by the maker of a product.

    You're starting to sound like the whiney little tattletale in the front of the room, George.
    shraven
    • So it's wrong to call someone out on hypocrisy?

      So it's wrong to call someone out on hypocrisy? Sounds like you know what AMD did is wrong but you're just upset someone is calling them out on it.

      "Tattletale"? That's a new one on me.
      georgeou
      • The funny thing about accusing someone of hypocrisy

        It's not wrong to call out hypocrisy. It is, however, hypocritical to only do so to one of the parties involved.

        Did AMD make a big stink about it first? Yes. Does that mean they're worthy of ridicule for being guilty of the same thing? You bet. But the fact of the matter is, Intel is now screaming just as loudly AFTER having been caught in a systematic attempt to mislead consumers, but before they've issued any sort of apology or made any attempt to clean up the dirty tricks their marketing team uses. This doesn't make AMD more contemptible than Intel; it just means they're both in need of new marketing folks.

        They both lied, they both squawked, and neither of them have any intention of doing anything differently. So roast them both; don't single out one or the other for special treatment. By the standard you seem to be endorsing, you're also guilty of cherry-picking *your* data...which destroys your credibility and opens you up for charges of hypocrisy.
        DevinAt3Sharp
        • Where did Intel complain to the media about AMD ethics?

          "It's not wrong to call out hypocrisy. It is, however, hypocritical to only do so to one of the parties involved."

          Where did Intel complain to the media about AMD ethics? You're grabbing at straws here.
          georgeou
          • Ahh deflection a tool of the magician

            I think he's limiting the conversation to you specifically George not Intel. You're the hypocrit in this article.

            Nothing wrong with saying AMD was hypocritical, but he makes a good point about you being equally hypocritical about AMD's ethics when you know Intel does the same thing.

            Now I agree on your point that when someone gets up in front of the camera and screams unethical in an industry that's been unethical for years on the subject. But I think AMD's saying, Intel you've crossed the line this time.

            3 times AMD george? What about 3 times George OU in Intel's defence?...for shame on legitimate journalists everywhere. Call a spade a spade and throw them both in jail. Whistleblowers everywhere gasp at your severe biasness on the subject.

            I'm admitably biased on AMD's side, however I think Intel's and you have a good point. But in your article you should have really brought the whole thing to a conclusion where we could quantify what the difference was. Are we talking a 2 hour video processing job is done in a half hour on one chipset vs. 1 1/2 hours? Or are we talking 2 hours vs 2 hours and 3 minutes?

            THAT'S where a journalist should be showing the point rather than putting salt in the open wound and making a celebrity boxing match out of the whole thing.
            rschror
      • And you're one to talk......

        As I've stated elsewhere, if you're punched in the nose enough times, you retaliate. Since Intel never apologized, or explained their faulty benchmarks, AMD decided to play by Intel's rules.

        Yes it's wrong, but it is understandable. But you are so insanely biased for Intel and against AMD, it's no surprise that you stick up for Intel, even though they were wrong to start this benchmark fight.
        linux for me
        • You admit that AMD is wrong but you're slamming me for not hitting Intel?

          You admit that AMD is wrong but you're slamming me for not hitting Intel? You sir are the definition of a zealot.
          georgeou
          • You ared the definition of "Media Whore"

            Your "reporting" makes Fox News look unbiased. You are a slave to your two
            primary masters, Intel and Microsoft. You have prostituted your name, and the
            reputation of Ziff so many times that you're no better than a street hooker. No,
            that's not fair; you're worse than that. Most hookers sell themselves to survive.
            You do it for self-promotion. You continue to publish one-sided, unresearched,
            regurgitated pablum that reads more like unedited press releases than anything
            remotely resembling journalism.

            At least a zealot believes in something. You just believe in selling yourself to the
            highest bidder, or the one you think will help George Ou the most.

            You ought to be ashamed, but you're so far past shame that you just disgust most
            people.
            JoeBob_z
          • That word, tender readers, is Who re

            (without the space, of course).
            I guess ZDnet thinks it's too jarring for your delicate constitution.
            JoeBob_z