Windows 8 Consumer Preview vs. Windows 7: Benchmarked
It's time to see how Microsoft's Windows 8 Consumer Preview stacks up against Windows 7. Can the upcoming operating system keep up with -- or even beat -- Windows 7, or does Microsoft still have work to do?
This is my second attempt at benchmarking the Windows 8 Consumer Preview. I attempted to benchmark the operating system soon after it was released back in February, but ran into troubles with graphics card drivers issues, and problems getting consistent results from a couple of the benchmark tools I was using. It seems that these issues have been ironed out, finally allowing me to complete the testing.
The hardware
The following hardware platform was used for benchmarking the two operating systems. The system was purpose-built for the job of benchmarking:
- Intel Core i7-2600K processor
- Crucial 4GB DDR3 1600 (PC3 12800) RAM
- EVGA 01G-P3-1460-KR GeForce GTX 560
- GIGABYTE GA-Z77MX-D3H motherboard
- Western Digital Caviar Black WD1002FAEX 1TB hard drive
- CORSAIR Enthusiast Series TX650 V2 650W power supply unit
Everything on the system was set to stock speeds, with no component overclocked.
For the tests I used two Western Digital Caviar Black WD1002FAEX 1TB hard drives from the same batch to eliminate the possibility of an update causing one drive to be faster than the other. One drive had installed Windows 7 Ultimate (SP1) 64-bit, and on the other I installed the Windows 8 Consumer Preview 64-bit. All drivers and updates were installed, along with all the software that would be needed for the tests. The drives were then defragmented using the Windows tool before the benchmarking was carried out.
The benchmark tests
Here's a rundown of the tests that were run on the two operating systems. I've chosen a mixture of real world and synthetic benchmark tests.
Each test was run three times and the results averaged.
- Boot time Measured using a handy tool called BootRacer. This measures both the time it takes to get to the logon screen and the time to boot to the desktop.
- Audio transcode time Transcoding an audio test file from WAV to MP3 format using iTunes. A measure of the operating system's ability to handle multimedia.
- Video transcode time Transcoding video test file from DVD to MP4 format using Handbrake. A measure of the operating system's ability to handle multimedia.
- PCMark 7 A benchmark run with PCMark 7. The industry standard PC test for CPU, HDD, SSD, memory, and graphics performance.
- 3DMark 11 A benchmark run with 3DMark 11. This is a set of six demanding benchmark test measuring the graphics performance of gaming PCs.
- FurMark A benchmark run with FurMark. This is a VGA stress test, GPU burn-in test and an excellent OpenGL benchmark. This is a very stressful benchmark and can damage or even destroy hardware if used incorrectly, and therefore I do not recommend running this tool on a system unless you know exactly what you are doing and fully understand the risks associated with it.
- Cinebench 11.5 A benchmark run with Cinebench 11.5. This is a real world cross, platform test suite that evaluates a computer's CPU and GPU performance capabilities.
- Heaven 3.0 A benchmark run with Heaven 3.0. This is a DirectX 11 GPU benchmark based on the advanced UNIGINE engine. Not only does this tool give us the maximum frames per second (FPS), it also records minimum frames per second, which is handy observing dips in performance during heavy load.
- Alien vs. Predator A benchmark run using the in-built benchmark tool available in Alien vs. Predator. The benchmark is run at 1920x1080 screen resolution with DirectX 11 enabled. This is a real world gaming test.
The results
Here are the results from each of the benchmark tests. Each test was run three times, and the average taken from all three runs.
No significant variance was seen between the three runs in any of the tests, a consistency that gives me confidence in the results.
Boot time
The Windows 8 Consumer Preview manages to shave a few seconds off the boot time and the time it takes to get to a usable desktop.
Audio transcode time
Audio transcoding using iTunes is slightly faster on the Windows 8 Consumer Preview. The difference is more noticeable when large files are being transcoded.
Video transcode time
Windows 8 Consumer Preview is comfortably faster at transcoding video using Handbrake. Again, this becomes more noticeable when transcoding larger files.
PCMark 7
Once again, the Windows 8 Consumer Preview is faster than Windows 7, but this time the difference is staggering. The difference was primarily made up by an increase in the "creativity" score, which measures multimedia and DirectX performance.
3DMark 11
There appears to be no significant difference between the two operating systems.
Furmark
Again, I saw no significant difference between the two operating systems..
Cinebench 11.5
OpenGL is slightly faster on Windows 7, but the difference between the two operating systems bears little significance.
Heaven 3.0
The interesting thing here are the minimum frames per second (FPS) scores. This test suggests that the minimum FPS score for Windows 8 Consumer Preview is lower than for Windows 7, showing that in-game FPS drops on the new operating system are greater than they are on Windows 7. This is significant because the lower minimum FPS drops, the worse the in-game experience becomes.
Alien vs. Predator
There appears to be no significant difference between the two operating systems.
The conclusions
We can draw some interesting conclusions from these benchmark results. The first and most obvious is that boot times have been cut quite nicely. Microsoft promised us this, and it seems that the promise has been delivered. We don't reboot out PCs anywhere near as often as we once did, but a fast boot up time is still appreciated.
Next there's the fact that, as far as the synthetic and gaming benchmarks go, the differences between Windows 7 and the Windows 8 Consumer Preview are negligible. This is quite interesting, because at this stage for Windows XP, Vista, and Windows 7, we were seeing poorer results for anything that relied heavily on the graphics system. It usually takes AMD and NVIDIA some time to optimize and perfect their drivers for a new operating system, with drivers having to mature for several months before we see similar performance between the new operating system and the old one.
With the Windows 8 Consumer Preview we seem to be getting very good results from the current crop of drivers, which is good news for gamers who are planning to make a swift switch to Windows 8 when it is released.
We're also seeing quite an improvement when it comes to audio and video transcoding. This is something I've come to expect from betas of Windows operating systems. It's an area that Microsoft seems to put effort into improving, and that trend continues with Windows 8.
The higher than expected PCMark 7 score for the Windows 8 Consumer Preview is interesting. Normally, I would be suspicious of such a difference between the two operating systems, and would be tempted to put it down to a bug with the benchmark tool. However, given that the bulk of the improvement was made up by an increase in the "creativity" score, and the fact we've seen an overall improvement in multimedia handling in other tests, I'm tempted to believe that this improvement is genuine.
The only result I am concerned about comes from the Heaven 3.0 test. The minimum FPS scores in this test seems to suggest that when the frame rate drops in the Windows 8 Consumer Preview, it drops further than it does for Windows 7. If this translates into real world gaming, it would mean a poorer visual experience under Windows 8.
However, it's quite likely that this is a driver issue, and something that will most likely be fixed when Windows 8 specific drivers come out.
All in all, I've very pleased with how Windows 8 is coming along. It is shaping up to be quite a capable operating system as far as performance is concerned.
Bonus: Overclocking with Windows 8
Over the past few weeks I've had a number of emails, mostly from gamers, asking me what Windows 8 is like when you overclock the hardware. There seems to be some concern that the operating system might make it harder to push your hardware to beyond what it was intended to do.
Given that I had a test-bed set up, I decided to see what I could do.
I was able to push the Core i7-2600K CPU up from the stock speed of 3.4GHz all the way to 4.7GHz without any problem at all. After some tinkering with multipliers and voltages, I managed to reach 4.9GHz with no stability issues and without the operating system throwing the towel in. At this point I decided to call it a day since I didn't want to put a smoking crater into the motherboard.
Next I turned my attention to the GeForce GTX 560. Using AfterBurner I managed to raise the core clock from the stock 810MHz to 975MHz, the shader clock from 1,620MHz to 1,954MHz, and the memory clock from 4,008MHz to 5,126MHz by raising the GPU voltage. Again, the Windows 8 Consumer Preview didn't complain.
These sorts of overclocks are what I'd expect to be able to achieve on Windows 7, so I would say that Windows 8 won't make overclocking any harder. In fact, the faster boot times makes it a lot easier, because if there's one thing you do a lot when overclocking, it's rebooting.
Related:
- Windows 8: Fragmentation of the hardware ecosystem is inevitable
- Windows 8: Media Center functionality likely to cost more
- Intel offers a sneak peek at a hybrid tablet ultrabook running Windows 8
- Bring Metro apps to Windows 7 to encourage developer growth
- Why Windows 8 won’t reimagine hardware that much
- Windows 8: Can we live without the desktop?
- Here’s what’s wrong with Windows 8
The conclusions
We can draw some interesting conclusions from these benchmark results. The first and most obvious is that boot times have been cut quite nicely. Microsoft promised us this, and it seems that the promise has been delivered. We don't reboot out PCs anywhere near as often as we once did, but a fast boot up time is still appreciated.
Next there's the fact that, as far as the synthetic and gaming benchmarks go, the differences between Windows 7 and the Windows 8 Consumer Preview are negligible. This is quite interesting, because at this stage for Windows XP, Vista, and Windows 7, we were seeing poorer results for anything that relied heavily on the graphics system. It usually takes AMD and NVIDIA some time to optimize and perfect their drivers for a new operating system, with drivers having to mature for several months before we see similar performance between the new operating system and the old one.
With the Windows 8 Consumer Preview we seem to be getting very good results from the current crop of drivers, which is good news for gamers who are planning to make a swift switch to Windows 8 when it is released.
We're also seeing quite an improvement when it comes to audio and video transcoding. This is something I've come to expect from betas of Windows operating systems. It's an area that Microsoft seems to put effort into improving, and that trend continues with Windows 8.
The higher than expected PCMark 7 score for the Windows 8 Consumer Preview is interesting. Normally, I would be suspicious of such a difference between the two operating systems, and would be tempted to put it down to a bug with the benchmark tool. However, given that the bulk of the improvement was made up by an increase in the "creativity" score, and the fact we've seen an overall improvement in multimedia handling in other tests, I'm tempted to believe that this improvement is genuine.
The only result I am concerned about comes from the Heaven 3.0 test. The minimum FPS scores in this test seems to suggest that when the frame rate drops in the Windows 8 Consumer Preview, it drops further than it does for Windows 7. If this translates into real world gaming, it would mean a poorer visual experience under Windows 8.
However, it's quite likely that this is a driver issue, and something that will most likely be fixed when Windows 8 specific drivers come out.
All in all, I've very pleased with how Windows 8 is coming along. It is shaping up to be quite a capable operating system as far as performance is concerned.
Bonus: Overclocking with Windows 8
Over the past few weeks I've had a number of emails, mostly from gamers, asking me what Windows 8 is like when you overclock the hardware. There seems to be some concern that the operating system might make it harder to push your hardware to beyond what it was intended to do.
Given that I had a test-bed set up, I decided to see what I could do.
I was able to push the Core i7-2600K CPU up from the stock speed of 3.4GHz all the way to 4.7GHz without any problem at all. After some tinkering with multipliers and voltages, I managed to reach 4.9GHz with no stability issues and without the operating system throwing the towel in. At this point I decided to call it a day since I didn't want to put a smoking crater into the motherboard.
Next I turned my attention to the GeForce GTX 560. Using AfterBurner I managed to raise the core clock from the stock 810MHz to 975MHz, the shader clock from 1,620MHz to 1,954MHz, and the memory clock from 4,008MHz to 5,126MHz by raising the GPU voltage. Again, the Windows 8 Consumer Preview didn't complain.
These sorts of overclocks are what I'd expect to be able to achieve on Windows 7, so I would say that Windows 8 won't make overclocking any harder. In fact, the faster boot times makes it a lot easier, because if there's one thing you do a lot when overclocking, it's rebooting.
Related:
- Windows 8: Fragmentation of the hardware ecosystem is inevitable
- Windows 8: Media Center functionality likely to cost more
- Intel offers a sneak peek at a hybrid tablet ultrabook running Windows 8
- Bring Metro apps to Windows 7 to encourage developer growth
- Why Windows 8 won’t reimagine hardware that much
- Windows 8: Can we live without the desktop?
- Here’s what’s wrong with Windows 8