Vista SP1 vs. XP SP2 - Benchmarked

Latest: Vista SP1 vs. XP SP2: Part Deux 

[UPDATE 02/18/08: Some people are concerned that in order to get an accurate measurement of boot time the system needs to be booted many times.  Let me assure you that the systems here under testing were rebooted more times than most normal PCs are over the course of several months.] 

Over the past few days I posted two sets of benchmarks comparing Windows Vista RTM with Vista SP1 (first post here, second here).  These posts generated a lot of feedback, and from reading this feedback it's clear that what many people are really interested in is not the performance differences between Vista RTM and Vista SP1, but between Vista SP1 and XP SP2.

A few days ago I posted in reply to several TalkBack comments that I wouldn't carry out these tests until XP SP3 is released.  There didn't seem any point.  That didn't satisfy the crowds who wanted to see Vista SP1 and XP SP2 go head to head.  So, to cut a long story short, the pestering continued and I eventually gave in.  So what follows are the fruits of nearly two whole days of work at the PC Doc HQ (the test was hampered by the death of a motherboard, something which rendered hours of work obsolete).

How does Windows Vista SP1 compare to Windows XP SP2?  Read on ...

The Test Rig -->

The Test Rig

The test ring that we'll be looking at here is one of the systems that I benched for the Vista RTM/SP1 tests.  It's my Pentium 950D system and is representative of the most "middle of the road" systems that I have.  Too slow a system and the results could be inconclusive, too powerful and that might skew the tests.

Here's the spec:

• Pentium 950D processor (3.4GHz)
• 4GB of RAM (being 32-bit, the OS recognizes a shade over 3GB)
• 2 x 250GB Maxtor hard drive

What I did was simple.  I took out the boot drive and found two identical 250GB drives.  I fitted one, installed Vista Ultimate 32-bit, set up the system and got it up and running.  Then I took out that drive and replaced it with the second drive, installed Windows XP SP2 onto this one and got this OS up and running.  Both systems are fresh and have all the latest drivers installed.

Note:  We abandoned the idea of using dual-boot early on because of the effect this could have on disk performance.

No optimizations were carried out other than defragging the drive and making sure that all background tasks had finished.

The Tests -->

The Tests

The tests are the same tests I've carried out before, consisting of moving, compressing and extracting a set of files.  I had two sets of test files:

• A single 664MB file
• A folder containing 500 files in 10 folders, total file size 995MB

The tests that were carried out using both sets of files were:

• Drive to drive copy
• Add to compressed folder (using Windows built-in compression)
• Extract files from compressed folder (using Windows built-in compression)
• Push files to a network share over a 1Gb Ethernet connection (destination system running Vista SP1)
• Pull files over a network share over a 1Gb Ethernet connection (source system running Vista SP1)
• Push files to an external USB hard drive
• Pull files from an external USB hard drive

I also measured the boot up time (I took this as the time from the boot loader to a usable desktop).

Each test was run four times, the poorest result was discarded and the average of the remaining three taken.

During the testing close attention was given to Microsoft's SP1 testing guidelines which were supplied to me prior to getting my hands on the SP1 code.

The Results - Tabulated -->

The Results - Tabulated

OK, with all that out of the way, here's the bit that you've been waiting for - the results.

The Results - Graphs -->

The Results - Graphs

For those of you who prefer to take in information visually, here's the data displayed in a series of graphs.

Conclusions and Caveats -->

Conclusions and Caveats

Note:  I didn't benchmark against XP SP3 because the EULA prevents the disclosure of benchmark results.  Also, since it's a beta, things can change between now and the final release.

So, onto conclusions.  Looking at the data there's only one conclusion that can be drawn - Windows XP SP2 is faster than Windows Vista SP1.  End of story.  Out of the fifteen tests carried out, XP SP2 beat Vista SP1 in eleven, Vista SP1 beat XP SP2 in two of the tests, and two of the tests resulted in a draw.

The best result for Vista SP1 was in the single file drive-to-drive copy, while the best result for XP SP2 was extracting multiple files from a compressed folder.  Given these results and taking into account the improvments that SP1 bought to Vista, if I was to go back and compare XP SP2 with Vista RTM, XP would have hammered Vista even harder.

Reminder:  These results apply to a single system and no optimizations were carried out to any of the systems.  Your mileage can, and probably will, vary.

I'll carry out some gaming benchmarks over the next few days.  Watch this space!

Thoughts?  Comments?

<< Home >>