Windows 8 vs. Windows 7: Benchmarked

Now that Microsoft's Windows 8 operating system has hit the release-to-manufacturing (RTM) stage, it's time to see how it stacks up against the incumbent Windows 7.

Can the upcoming operating system keep up with -- or even beat -- Windows 7, or does Microsoft still have work to do to optimize performance?

Note that the RTM version of Windows 8 is the version that is sent to OEMs to load onto new systems. 

I have previously benchmarked both the Windows 8 Consumer Preview and Windows 8 Release Preview releases.

The hardware

The following hardware platform was used for benchmarking the two operating systems. The system was purpose-built for the job of benchmarking:

• Intel Core i7-2600K processor
• Crucial 4GB DDR3 1600 (PC3 12800) RAM
• EVGA 01G-P3-1460-KR GeForce GTX 560
• GIGABYTE GA-Z77MX-D3H motherboard
• Western Digital Caviar Black WD1002FAEX 1TB hard drive
• CORSAIR Enthusiast Series TX650 V2 650W power supply unit

Everything on the system was set to stock speeds, with no component overclocked.

For the tests I used a Western Digital Caviar Black WD1002FAEX 1TB hard drive with the Windows 8 RTM 64-bit installed on it. All drivers and updates were installed, along with all the software that would be needed for the tests. The drive was then defragmented using the Windows tool before the benchmarking was carried out.

Data related to the Windows 8 Consumer Preview and Windows 7 was collected from the a benchmark test of Windows 8 I carried out in April and the Release Preview data dates back to June.

The benchmark tests

Here's a rundown of the tests that were run on the three operating systems. I've chosen a mixture of real world and synthetic benchmark tests.

Each test was run three times and the results averaged.

• Boot time 
  Measured using a handy tool called BootRacer. This measures both the time it takes to get to the logon screen and the time to boot to the desktop.
• Audio transcode time 
  Transcoding an audio test file from WAV to MP3 format using iTunes. A measure of the operating system's ability to handle multimedia.
• Video transcode time 
  Transcoding video test file from DVD to MP4 format using Handbrake. A measure of the operating system's ability to handle multimedia.
• PCMark 7 
  A benchmark run with PCMark 7. The industry standard PC test for CPU, HDD, SSD, memory, and graphics performance.
• 3DMark 11 
  A benchmark run with 3DMark 11. This is a set of six demanding benchmark test measuring the graphics performance of gaming PCs.
• FurMark 
  A benchmark run with FurMark. This is a VGA stress test, GPU burn-in test and an excellent OpenGL benchmark. This is a very stressful benchmark and can damage -- or even destroy -- hardware if used incorrectly, and as such I do not recommend running this tool on a system unless you know exactly what you are doing and fully understand the risks associated with it.
• Cinebench 11.5 
  A benchmark run with Cinebench 11.5. This is a real world cross, platform test suite that evaluates a computer's CPU and GPU performance capabilities.
• Heaven 3.0 
  A benchmark run with Heaven 3.0. This is a DirectX 11 GPU benchmark based on the advanced UNIGINE engine. Not only does this tool give the maximum frames per second (FPS), it also records minimum frames per second, which is handy observing dips in performance during heavy load.
• Alien vs. Predator 
  A benchmark run using the in-built benchmark tool available in Alien vs. Predator. The benchmark is run at 1920x1080 screen resolution with DirectX 11 enabled. This is a real world gaming test.

The results

Here are the results from each of the benchmark tests. Each test was run three times, and the average taken from all three runs.

No significant variance was seen between the three runs in any of the tests, a consistency that gives me confidence in the results.

Boot time

The Windows 8 RTM has managed to shave another second off the total boot time to a usable desktop, although the boot time to logon remains the same as it was for the Release Preview.

Audio transcode time

There was no noticeable difference between Windows 8 RTM and the Windows 8 Release Preview, though both are significantly faster than Windows 7.

The improvement is more noticeable when large files are being transcoded.

Video transcode time

Windows 8 RTM shows a slight improvement over Windows 8 Release Preview, which was itself comfortably faster at transcoding video using Handbrake for Windows 7.

Again, this becomes more noticeable when transcoding larger files.

PCMark 7

Once again, the Windows 8 RTM ends up with a far superior score than Windows 7. The bulk of the improvements were primarily made by a healthy increase in the "creativity" score, which measures multimedia and DirectX performance.

However, the difference between the Windows 8 RTM, Windows 8 Release Preview and Windows 8 Consumer Preview is negligible.

3DMark 11

There appears to be no significant difference between the operating systems.

Furmark

This test showed no significant difference between the operating systems.

Cinebench 11.5

OpenGL is still slightly faster on Windows 7, but the difference between the operating systems bears little significance.

Heaven 3.0

Back when I tested the Windows 8 Consumer Preview, I was concerned by the results that suggested that the minimum FPS score was lower than for Windows 7, which indicated that in-game FPS drops on the new operating system were greater than they are on Windows 7. I considered this significant because the lower minimum FPS drops, the worse the in-game experience becomes.

However, with the Windows 8 Release Preview, Microsoft tightened up the code and bought performance to that of what you'd expect from Windows 7. With the RTM release, it seems that Microsoft -- quite possibly with the help of more mature graphics card drivers -- have improved the score even more.

This is good news for gamers hoping to make the switch to Windows 8.

Alien vs. Predator

There appears to be no significant difference between the operating systems.

The conclusions

We can draw some interesting conclusions from these benchmark results. The first and most obvious is that Microsoft has obviously worked hard to cut system boot times, as Microsoft previously promised. 

We don't reboot our PCs anywhere near as often as we once did, but a fast boot up time is still appreciated, and a PC that arrives at the logon screen or desktop quickly makes a good impression on both Microsoft and OEMs.

Hybrid boot, UEFI firmware and better use of sleep will make startup under Windows 8 even faster.

Next there's the fact that, as far as the synthetic and gaming benchmarks go, the differences between Windows 7, the Windows 8 RTM, the Consumer Preview and the Release Preview are negligible. It usually takes AMD and NVIDIA some time to optimize and perfect their drivers for a new operating system, with drivers having to mature for several months before we see similar performance between the new operating system and the old one.

This time around it seems that things have settled down quickly and that we're seeing performance that is on a par with a mature operating system. We can assume that as time goes on the graphics card makers will be able to squeeze more performance out of the operating system.

This is good news for anyone who is planning to make a swift switch to Windows 8 but also for those who want the best performance possible from their hardware.

We're also seeing quite an improvement when it comes to audio and video transcoding. This is something I've come to expect from Microsoft. It's an area that Microsoft seems to put effort into improving, and that trend continues with Windows 8. As we take more photos and video and handle more content, the ability to process them faster is welcome all round.

The higher than expected PCMark 7 score for the Windows 8 RTM, Release Preview and Consumer Preview compared to Windows 7 is interesting. Normally, I would be suspicious of such a difference between, and would be tempted to put it down to a bug with the benchmark tool. However, given that the bulk of the improvement was made up by an increase in the "creativity" score, and the fact we've seen an overall improvement in multimedia handling in other tests, I believe that this improvement is indeed genuine.

When I tested the Windows 8 Consumer Preview back in April I was concerned about the results I saw from the Heaven 3.0 test. The minimum FPS scores I got from this test seemed to suggest that when the frame rate dropped in the Windows 8 Consumer Preview, it dropped further than it did under Windows 7.

Translating this into real world gaming, it would mean a poorer visual experience under Windows 8. However, it seems that this is an area that Microsoft has been working on, and in the Windows 8 RTM version I'm not seeing the same frame rate drops as I was seeing with the Windows 8 Consumer Preview.

From a performance perspective, I've very pleased with the way that Windows 8 has turned out. While there are no major performance differences between the Windows 8 Release Preview and the newly released Windows 8 RTM version, performance seems solid, and in areas where the platform lagged behind Windows 7, Microsoft seems to have put in the effort to close the gap.