Hammering a dual quad-core Xeon system

Hammering a dual quad-core Xeon system

Summary: The other day I wanted to test out the viability of simultaneously transcoding six DVDs to six separate .MP4 files suitable for my iPod touch. So I loaded a few software utilities onto a dual quad-core Xeon system and put it through its paces. How did this system cope with this workload?

SHARE:

The other day I wanted to test out the viability of simultaneously transcoding six DVDs to six separate .MP4 files suitable for my iPod touch. So I loaded a few software utilities onto a dual quad-core Xeon system and put it through its paces. How did this system cope with this workload?

You might be wondering why I tried this out. It's because I'm planning to upgrade one of my systems to transform it into a monster DVD ripping machine suitable for tackling my 600+ DVD library but I wanted to test out a few theories before spending money.

So, here's the test. One dual quad-Xeon based system (E5472 Harpertown Core, 3.00GHz) running Windows Server 2008. Onto this system I loaded AnyDVD and CloneDVD Mobile. I took an .ISO of a DVD that I have and made three duplicates of this on one drive, and loaded each instance into a virtual drive. I then took a DVD and put this into the real optical drive, giving me six drives holding six DVDs to transcode. Then, to simulate the worst case scenario I set up CloneDVD Mobile to save the .MP4 files to another hard drive (so in essence the .ISO files are being read from one drive and ripped to another drive).

Hammering a dual quad-core Xeon system

So, how did this work out? Well, I learned a few useful lessons:

  1. Transcoding is heavy on CPUs and six transcoding operations running simultaneously can push both all eight cores on the Xeons to 100%
  2. Despite the workload, the transcoding of the six DVDs was done in 42 minutes (previous tests I'd done on a Core 2 Extreme QX9650 system did two files in 20 minutes)
  3. When ripping to .ISO files rather than transcoding to .MP4, the process is much less CPU intensive and the bottleneck then becomes the transfer rate of the hard disks
  4. It is possible to do the ripping and transcoding in two steps - first rip the DVD to .ISO files and then mount these virtually and transcode them to .MP4.

With this in mind I've ordered the components I need to upgrade a system to make it ideal for ripping and transcoding. The system will consist of:

  • A Q9300 2.5GHz quad-core processor. This 45nm piece will consume less power and run cooler than the Q6600 and it also benefits from supporting SSE 4.1 instruction set (which will boost transcoding performance when using compatible applications).
  • A GIGABYTE GA-EP35-DS3R motherboard fitted with 8 SATA ports.
  • 4 x SATA DVD drives. I'm choosing to fit four for a couple of reasons - first, the case I'm using can only hold four, and secondly I've chosen to devote a core to each rip/transcode operation.
  • 5 x 750GB hard drives in a RAID 5 array.

More on the build of this system later this week.

Topics: Enterprise Software, Hardware, Mobility, Processors

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

36 comments
Log in or register to join the discussion
  • Question...

    If you run just one instance, will it still multithread well enough to consume all 8 cores? If so, how fast does it complete that single task?
    Stuka
    • Multi-threading

      The one part of the process that cannot be threaded is the
      etching. That has to be a continuous stream and cannot block
      for a buffer refill. At best, O(n) in complexity (unless the
      encoding has higher complexity, I think O(n^2) and O(n log n)
      would play the role of the usual suspects here).

      Fetching and filling the buffer could be threaded, however, care
      has to be taken to be sure that two threads filling a buffer do
      not do so out of sequence.

      It seems to me that having a single thread at a time fetch and
      buffer a block simplifies the synchronization. The controlling
      program dispatches a thread to get a block (maybe 1/4 a
      buffer). Upon return a new thread is dispatched for fetching
      and a queue of fetched blocks is begun. When the buffer has
      been consumed sufficiently, the first thread in the queue
      offloads its block and dies. At the moment the queue signals it
      has a free slot, a new thread is created and dispatched for a
      fetch.

      As you can see, the threads do block to the degree they fetch
      and arrive faster than a block's worth of the buffer can be
      consumed by the etcher. It's the etcher that drives the time it
      takes to burn.

      Having a single processor burn two DVDs will increase the
      probability that one of the etchers will be starved for resources
      and the burn fails. (This probability increases if the burning
      speed increases.) Adding more burners increases the
      probability of one or the burns failing. There is, then, an
      optimal arrangement where the number of successful burns in
      unit time is maximized. Let's say a single burner is 99%
      successful, in 100 units of time, we would expect 99 successes.
      Let's say two burners/ one processor is 77% successful: 154
      discs in 100 units of time. Three burners at 54% successful:
      162 discs in 100 units of time. (Meanwhile one threw out 1, 46,
      and 138 bad discs, respectively, during those runs.)

      Well, that was fun.
      DannyO_0x98
  • except those MPEG-2 still take up 2-3 times more HD space...

    as the MPEG-4 for the same quality... if you are actually
    going to take the time to rip all your DVDs... for me it was
    about 400 DVDs, you might at well take the time and do it
    with a modern codec... i chose H.264 which even does little
    better than MPEG-4.
    doctorSpoc
    • Fair enough

      If you want to keep a long term copy of your DVDs on hard drive for some other purpose than just transferring them to a portable device then yes, transcoding is a good idea. If you are simply doing this because your portable media player is incapable of playing MPEG2 files though, do the transcoding knowing that your media player just cost you a ton of extra time and money. Buyer beware! :)
      NonZealot
    • MPEG4 is far more superior than MPEG2

      MPEG4 came out just a few years ago,
      and it is by far more superior MPEG2.

      MPEG4 is lossless compression, and its
      files are way smaller in size than
      MPEG2's. The colors are better too.

      MPEG4 comes in 3 extensions:
      .mp4, .m4a (for audio only), and .m4v
      (for video).

      I transcoded a 44.1/16-bit/stereo .wav
      file to mp4 @ 320kbs/16-bit/stereo.
      Then I compared the two files side by
      side in SoundForge, using two very high-
      end plugins: PAZ Analyzer (by Waves)
      and Spectral Frequency Analyzer. There
      was difference in sound quality ... the
      mp4 audio sounded just the same as the
      uncompressed .wav file. Nothing was
      lost. Everything sounded and LOOKED
      the same.

      However, mp3 audiopart of MPEG2
      (mpeg2, layer 3). It is very lossy .... on
      both mp3 audio and mpeg2 movie files.
      MP3 audio will have a lot of the sound's
      dynamics and frequencies chopped off
      the top and bottom end, making it
      sound less bright and less bassy. Mpeg2
      videos usually appear more blurred
      and/or grainy, with a lot more loss of
      color information. Transitions between
      scenes can suck too. It depends.

      MPEG2 is very very old technology ....
      been around for more than 20 years (or
      more, I guess).

      There's a reason why MPEG4 is now the
      new standard. Hence that a lot of HD
      content, Bluray DVD's, and other new
      media players (hardware and software)
      are now using MPEG4 formats.

      About H.264 ... that means how it's
      compressed ... it's one kind of
      compression-algorithm used to
      compress and transcode movies into
      mpeg4 files .... and it's only for MPEG4!
      You cannot use it to compress movies to
      MPEG2. There are various other kinds of
      optional compression-algorithms to
      choose from to make MPEG4 movie
      files.
      OmarHash
      • Lossless compresion is ....

        ... an oxymoron. It doesn't exist!
        ShadeTree
        • Gee does that mean all of my zip files are ...

          unrecoverable ???
          mrlinux
      • No such thing...

        Its either lossless, or compressed. Lossless would be a straight DV file, which is ginormous. Compression shrinks the file by removing elements of it.
        Stuka
        • Wrong again...

          When it comes to compressing files, they
          can be lossy or lossless .... one of the two
          .... look this up. MPEG4 is lossless.

          Even standard DVD movies are
          compressed to death in MPEG2 format
          which is lossy.

          MPEG4 is lossless and its the best format
          for HD content, especially on bluray
          players and other hardware/software
          media players which can play media-files
          in this format. Even its streaming
          media/data is way superior than anything
          out there.

          Uncompressed media data are usually
          referred to as "native formats" and NOT
          referred to "lossless." Native media
          formats are large in size and raw ....
          which are then transcoded to other
          formats and compressed to make the files
          smaller.

          Lossless refers to compression, as there
          are many intermediate codec algorithms
          which do that.

          According to wiki:
          "Lossless data compression is a class of
          data compression algorithms that allows
          the exact original data to be
          reconstructed from the compressed data.
          This can be contrasted to lossy data
          compression, which does not allow the
          exact original data to be reconstructed
          from the compressed data.
          ...Lossless compression is used when it is
          important that the original and the
          decompressed data be identical..."

          "...[MPEG4 adds] new features such as
          (extended) VRML support for 3D
          rendering, object-oriented composite files
          (including audio, video and VRML objects),
          support for externally-specified Digital
          Rights Management and various types of
          interactivity."

          It can also play back AC3 audio (audio
          format for 5.1 playback: 6 discrete audio
          tracks/channels for front-left, front-right,
          center, surround/rear-left, surround/rear-
          right, and LFE (lower frequencies or the
          bass).

          MPEG4 also can store multiple chapter
          marks. So when you transcode a DVD
          movie to MPEG4, make sure to check
          "include chapter marks."

          MPEG4 also can store subtitles. And much
          more.

          I suggest that you download an audio
          mp4 podcast. Make sure that you
          download the right one and has
          embedded chapter marks, pictures, video
          clips, and any other interactive features.
          Usually such information should be stated
          somewhere. Play the podcast in iTunes.
          iTunes and Quicktime are the only players
          lets you take full advantage of mpeg4
          media. See for yourself.
          OmarHash
          • Uh... no...

            First, MPEG-4 can use a large aray of audio and video codecs. MPEG-4 is the wrapper. MPEG doesnt support lossless audio, such as FLAC, WMA Lossless, Apple Lossless, etc. And it doesnt support lossless video formats either. Not to mention creating a lossless file of an DVD would create a file that is about twice the size of the original.

            When you compress a file, you remove data. Therefore you are losing data. So it is not lossless. I don't care what wikipedia says. A file is either lossless, or compressed. PERIOD.
            Stuka
          • you are actually dead wrong about lossy and lossless compression...

            think about this... when you zip a word file to email it to a
            friend is there data lost when the person opens it on the
            other side? NO! this is lossless compression.

            this is how a format like Apple Lossless works... think zip
            file... no data is lost, but the stored file is a smaller, more
            compact version of the original. Even thought the Apple
            lossless file is larger than an AAC file or MP3 (lossy
            compressed), the original WAV file is MUCH larger than the
            Apple Lossless file.

            as the file is being played back it is uncompresses (think un-
            zipped) and the result is the data from the played back file is
            identical to the original uncompressed WAV file even though
            the stored file (Apple Lossless) is much smaller.

            in lossless compression no data is lost, they just use different
            techniques to store the data in a more compact manner...
            removing repetition and recognizing patterns etc.. e.g. it
            would take up much less space to write a notation like
            "100*0" than it does to actually write out 100 zeros, or to
            write 6 a character color definition FFFFFF (hexadecimal) than
            to write a 9 character color definition 255255255 (decimal)
            even though they represent the EXACT same thing.. i.e. no
            data loss..

            the only caveat is that the reading of these files is much more
            CPU intensive... you'll notice when playing an MPEG-4 or
            H.264 file you'll use more CPU (and battery if you're on a
            laptop) than playing a MPEG-2.

            so you are wrong, a file can be (and are) lossless AND
            compressed... in fact one of your examples.. Apple Lossless is
            in fact an example of a file format that is lossless AND
            compressed.
            doctorSpoc
          • mpeg4 links

            http://en.wikipedia.org/wiki/MPEG-4_SLS

            http://en.wikipedia.org/wiki/Audio_Lossless_Coding
            OmarHash
          • Well you are way wrong....

            2 Examples

            1) zip files are compressed data, and they are completely lossless (Otherwise they would be useless)

            2) http://en.wikipedia.org/wiki/Lossless
            Video

            * Animation codec
            * CorePNG
            * FFV1
            * JPEG 2000
            * Huffyuv
            * Lagarith
            * MSU Lossless Video Codec
            * SheerVideo
            mrlinux
          • GPAC Osmo4 player

            Actually, the GPAC Osmo4 player (http://gpac.sourceforge.net/index.php) and a java-based MP4 player from IBM (I forget the name) are more capable wrt MPEG-4 systems than iTunes and Quicktime.
            e-man_z
          • More info

            The IBM player can be downloaded from their website on MPEG4 technologies at http://www.research.ibm.com/mpeg4/index.htm. It's a very basic player wrt everything other than MPEG-4 systems stuff. It's used in some MPEG-4 systems demonstrations at http://www.research.ibm.com/mpeg4/Demos/DemoSystems.htm. The web page includes the player, which is java-based.

            A better link for the GPAC Osmo4 player is http://gpac.sourceforge.net/player.php. They have demos at http://gpac.sourceforge.net/home_screens.php. Of course, you need Osmo4 to play them -Quicktime can't handle most of them.
            e-man_z
        • Counter-examples

          Huffman Trees, ZIV. (.zip files, .gif).

          Not useful for content that doesn't have long stretches of
          repeated elements or elements which may be identified as having
          higher statistical occurrence. Still, compression without loss.
          DannyO_0x98
          • GIF isnt lossless

            GIF is limited to 256 colors (removing most of the data fromt he original image) and it uses dithering to fill in the gaps for the rest of the data that it removed.
            Stuka
          • GIF sure isn't lossless, but the .zip file sure is... right?

            lossless compression works in this manner... Apple Lossless
            works in this manner. the file is just uncompressed (like
            unzipping) at playback time and the played back file is
            identical to the much larger .WAV file. this playback is much
            more CPU intensive though as i wrote in my other post.
            doctorSpoc
          • You are correct about zip

            I stand corrected there. Zip does a shorthand of sorts for the data contained in the file. So the data is not removed from the file, it is simply changed to use less space.
            Stuka
        • Lossy vs. lossless

          It is either compressed or uncompressed. If it is compressed, it is either lossy or lossless. http://en.wikipedia.org/wiki/List_of_codecs.

          Free Lossless Audio Codec (FLAC) is lossless - the original data can be recreated. It is compressed, because the information takes up less space.
          e-man_z