ie8 fix
madison

Hardware 2.0

Adrian Kingsley-Hughes

Vista RTM vs. Vista SP1 - Office 2007 benchmarking

By | February 26, 2008, 8:33am PST

Summary: Enough with benchmarking the OS - let’s see if Office 2007 is any faster on Vista SP1.

Enough with benchmarking the OS - let’s see if Office 2007 is any faster on Vista SP1.

The test

The tests will be carried out on the AMD Spider platform that I have set up in the lab (Phenom 9700, Radeon 3850 graphics card, 2GB of RAM …).  I’ve used this system as the platform for a number of benchmarks I’ve run over the past week (for a full spec, see this post).

As for the tests, we took two identical images of Vista - one RTM, one SP1.  We then loaded Microsoft Office 2007 Professional onto the system and applied all the patches (including Office 2007 SP1).  The system was then defragged and rebooted several times. 

Afterwards we downloaded and installed DMS Clarity Studio software which includes the OfficeBench software.  This application comes with a comprehensive set of Microsoft Office test scripts.  These test scripts, while not perfect (no benchmark solution is) are pretty good and simulate a number of real-world tasks that users might carry out in Microsoft Office.  The metric that we’re interested in getting from this benchmark is how long OfficeBench takes to execute the test scripts.

All tests were run five times under no load and five times under load (here OfficeBench runs a small Windows Media Video in the background), and we rebooted the system between each run.  The tests were duplicated on Vista RTM and Vista SP1.

Because the results were so well grouped for this test, no results were discarded and times taken to execute the scripts were averaged.  The lower the time, the faster the scripts completed and the better the result.

The Results and Conclusions –>

Adrian Kingsley-Hughes is an internationally published technology author who has devoted over a decade to helping users get the most from technology.

Disclosure

Adrian Kingsley-Hughes

All opinions expressed on Hardware 2.0 are those of Adrian Kingsley-Hughes. Every effort is made to ensure that the information posted is accurate. If you have any comments, queries or corrections, please contact Adrian via the email link here. Any possible conflicts of interest will be posted below. [Updated: February 23, 2010] - Adrian Kingsley-Hughes has no business relationships, affiliations, investments, or other actual/potential conflicts of interest relating to the content posted so far on this blog.

Biography

Adrian Kingsley-Hughes

Adrian Kingsley-Hughes is an internationally published technology author who has devoted over a decade to helping users get the most from technology -- whether that be by learning to program, building a PC from a pile of parts, or helping them get the most from their new MP3 player or digital camera.

Adrian has authored/co-authored technical books on a variety of topics, ranging from programming to building and maintaining PCs. His most recent books include "Build the Ultimate Custom PC", "Beginning Programming" and "The PC Doctor's Fix It Yourself Guide". He has also written training manuals that have been used by a number of Fortune 500 companies.

Adrian also runs a popular blog under the name The PC Doctor, where he covers a range of computer-related topics -- from security to repairing and upgrading.

19
Comments

Join the conversation!

Just In

It's considered
boogie2277@... 29th Feb 2008
I really don't know the benefits of 07 vs 03, But like most new products, It's really been hyped! and thats how I kinda feel about it.
I've read bits and pieces of how intergrated it is into windows and ease of use. and the online features.
I guess I already have features I need, 2003 has good online feaures and tutorials that I've used regulary to educate myself with it's features. Is there any talk of not supporting 2003, I'm really not sure about long term.
rather It's work or play, software and hardware companies get into my pockets regulary.
thanks itpro_z for your view. I think were on the same page.
0 Votes
+ -
office 2007
g_keramidas@... 26th Feb 2008
not totally pertinent to the discussion, but office 2007, specifically excel, is slower than office 2003 on xp or vista.

i have excel code that takes twice as long to execute in office 2007 than it does in office 2003. the os doesn't seem to have much of an impact on the running time, no matter if it is xp, vista rtm or vista sp1 on x86 or x64.x64 may be slightly faster, but not significant enough to worry about.
0 Votes
+ -
Why don't you bench on an Intel penryn desktop system too?
0 Votes
+ -
Given that the real bottleneck using Office is the speed at
which one types, your point about the arbitrariness of these
benchmarks is clearly, absolutely on point.

The closeness of the scores for loaded/not loaded in SP1
suggests to me that there is pre-fetching or pre-loading of
Office taking place.

If you aren't sick of this stuff, maybe a comparison of a
command-line batch program (perhaps ImageMagick
processing a directory of images) may provide an interesting
basis of comparison. Added bonuses: as a command line
program, it can be easily scripted and looped and, also, it isn't
Microsoft's software, so that means no unequal opportunity
grooving.
0 Votes
+ -
Agreed
CobraA1 26th Feb 2008
Agreed. Computers have been more than powerful enough to handle Office applications for years, and frankly these days the limiting factor is indeed how fast you can type. These tests are rather pointless.
0 Votes
+ -
And also...
CreepinJesus 27th Feb 2008
Boot-up/shutdown times, IE load times, things like that happy

I'm not fussed if it takes .4 of a second longer to save a Word document. I am fussed, however, if it takes a minute longer to boot up the system...
0 Votes
+ -
It would be interesting to see (yes, I realize I'm suggesting yet ANOTHER benchmark) how Office 2007 performs on XPsp2 versus Vistasp1. I think that right there is a great "real world" example of how businesses can optimize their ROI.

If, for example, XP outshines Vista and Office is the majority application for a business, then it would make no "real world" sense to migrate a business to Vista as all it would do is slow down the primary task.

I think Office is an excellent tool as it is what the majority of business use focuses on. This would also help stop the Vista versus XP issue and focus on the task...which is what most businesses care about.

Regardless of which OS wins, this would determine the need based on functionality and ROI for the business rather than just what the sales pitch keeps telling us to do...regardless of whether the answer is Vista or XP.
0 Votes
+ -
AFAIK, it's the same Office in either OS, and I don't think Microsoft turns on or off features based on the OS, so I seriously doubt there's going to be any functionality concerns.
0 Votes
+ -
2007 vs.....
rhonin 26th Feb 2008
Have 2007 on one laptop and 2003 on the other.
While I really like the layout and changes to 2007, there are some integrated spreadsheets I don't run on 2007 anymore due to the time it takes to load and run.

Too bad....
0 Votes
+ -
this test is pure FUD
qmlscycrajg 27th Feb 2008
this test is pure FUD
0 Votes
+ -
Vista SP1 vs Windows 2008
booknut 27th Feb 2008
I'm seeing a lot of buzz about how much faster (better, etc.) Windows 2008 is as a workstation than Vista SP1. Since they both use the same core, I'm wondering if Adrian could provide some benchmark numbers verifying/disproving this notion.
0 Votes
+ -
it's a placebo effect.
rtk 27th Feb 2008
Vista and Win2k8 server use exactly the same kernel, anyone who claims it's remarkably faster than Vista is making it up.
0 Votes
+ -
All of that "buzz"...
itpro_z 28th Feb 2008
...is just that. Windows 2008 is not out yet, and is a server OS anyway. There may very well be a workstation version of it, which will be mainly Vista stripped down to run lean and clean, with a few enhancements to make it better suited for high end workstation use. Expect it to be priced accordingly. You can do the same right now with Vista if you are willing and able to custom configure both your hardware and software.
0 Votes
+ -
Adrian, Please resond to this
boogie2277@... 27th Feb 2008
I'm just an average, everyday PC user, I read your thoughts and comments because of all the info and user comment's.
Just some sort of a guideline.
I'm a manager of a truck part's retail store and my wife is an operations manager over a large car parts distribution chain.
I have 2 wired, 2 wireless desktops,2 part time notebooks.
specs are anywhere from P4,core 2,4000 X2 and at least 2 gigs in all.
my question is there a performance impact on my lower systems if I upgrade to office 2007? Because I have no problems running office 2003. All input by readers too please!
P.S.P4, and notebooks has XP pro, all others Vista Ultimate.
Thanks
0 Votes
+ -
Contributr
Before I answer ...
Adrian Kingsley-Hughes 27th Feb 2008
... lemme throw this question at you ...

What specifically in Office 2007 do you want?
0 Votes
+ -
Consider this
itpro_z 28th Feb 2008
If your systems are working well for your, why would you want to mess with them? Software is not like a car, where a new model might have dramatic improvements in some areas over an older model (safety, fuel economy, etc). You use your software for specific tasks, and those tasks will not change if you switch from Office 03 to 07. You probably have better things to do with your time and money. I still have many users on Office 2000. When I need to replace their computer, I will upgrade their Office, but probably not before. Depending on the user, they may get Open Office rather than MS Office anyway.
0 Votes
+ -
It's considered
boogie2277@... 29th Feb 2008
I really don't know the benefits of 07 vs 03, But like most new products, It's really been hyped! and thats how I kinda feel about it.
I've read bits and pieces of how intergrated it is into windows and ease of use. and the online features.
I guess I already have features I need, 2003 has good online feaures and tutorials that I've used regulary to educate myself with it's features. Is there any talk of not supporting 2003, I'm really not sure about long term.
rather It's work or play, software and hardware companies get into my pockets regulary.
thanks itpro_z for your view. I think were on the same page.
0 Votes
+ -
I also run a small computer business 15 systems and 60 paying clients, and here are our findings...

Cost of training huge
VISTA (ULT) this slowed down out daily tasks
Interface was clumsy and hard to navigate
Office 2007 PRO this was slow
Macros and Templates from Office 2003 a large number would have had to be rewritten
Ribbon intrerface hated by our users and clumsy
After testing for 2 weeks on 5 systems and one we sent out for end users to test in their offices was DUMPED.
MADE our business unproductive and uneconomic
The same findings by our clients.
WE will remain with XP PRO and Ofice 2003 and if MS discountinues support we are evaluating LINUX FEDORA 8 and an an online office suite ZOHO both are looking good
0 Votes
+ -
hard to believe you
eggmanbubbagee@... 27th Feb 2008
If they found it that hard to adapt to Vista and 07 you expect us to believe they can adapt to Linux? Ha ha
0 Votes
+ -
A different perspective
itpro_z 28th Feb 2008
I work in an IT department with a few hundred users on our network. I can't speak for Ultimate, as I personally don't see any applications where I would recommend it, but Business has worked very well for us, to the point where we are no longer using XP on any new installs. The users we have converted rave about the improvements in Vista's GUI. As for productivity, that comes down to how we set up the systems. Most users use the OS to run programs and open files, and if I set up the machine properly, there is little change. The improvements in the GUI, security, multitasking, and stability have been dramatic, though.

We have not made a decision to move to Office07, but do have a few departments that have switched on their own. After the usual "getting used to" period, those users on 07 tell me that they find it easier to accomplish common tasks, implying that their productivity has gone up. Most of them have gone through several Office upgrades, so this change has been relatively easy. We are also using Open Office on more and more machines, and probably will end up with a mix, using Office07 for power users and OO for many others.

Your clients look to you for advice and support. If you present a negative attitude about new software, and you are not prepared to properly support it, then I am not surprised that you get negative feedback. I handled Vista like I have every version of Windows: I adopted it fairly early, leaned its quirks and intricacies, got up to speed for supporting it, and moved ahead. As a result, my users have had a positive experience with Vista, and I have had no "roll backs" like many of you keep talking about.

You can stay with XP for a while, but the market moves on. I preferred 2K over XP, and still do, but 2K is no longer viable for many applications. XP will eventually fade away, just as every version of Windows before it has. If you are unwilling to learn to support Vista, then you should invest your time learning Linux, or perhaps the Mac. Making the change is not easy, either, so you will still have your work cut out for you.

Join the conversation!

Formatting +
BB Codes - Note: HTML is not supported in forums
  • [b] Bold [/b]
  • [i] Italic [/i]
  • [u] Underline [/u]
  • [s] Strikethrough [/s]
  • [q] "Quote" [/q]
  • [ol][*] 1. Ordered List [/ol]
  • [ul][*] · Unordered List [/ul]
  • [pre] Preformat [/pre]
  • [quote] "Blockquote" [/quote]
ie8 fix
Click Here
ie8 fix

The best of ZDNet, delivered

ZDNet Newsletters

Get the best of ZDNet delivered straight to your inbox

Facebook Activity

White Papers, Webcasts, & Resources
ie8 fix
ie8 fix