Microsoft Office versus Open Office shootout

[Updated 9/15/2005 12:00PM: Added Open Office Beta2 results] When I started this blog questioning the wisdom of XML, it started a storm of comments and was followed up by fellow bloggers John Carroll and Joe McKendrick.  Since my response "Proof that XML is extremely bloated" appeared last week, I got a lot of talkback complaining that I was only testing Microsoft Excel and not Open Office Calc.

[Updated 9/15/2005 12:00PM: Added Open Office Beta2 results] When I started this blog questioning the wisdom of XML, it started a storm of comments and was followed up by fellow bloggers John Carroll and Joe McKendrick.  Since my response "Proof that XML is extremely bloated" appeared last week, I got a lot of talkback complaining that I was only testing Microsoft Excel and not Open Office Calc.  ZDNet reader Yagotta B. Kidding took it upon himself to do some testing with Open Office but he stopped testing when he reached a file half the size of my 16-sheet XML file, yet his results were even slower than my Excel results.  I requested that Kidding provide the results for the identical 16-sheet file, but he declined and wouldn't provide a specific reason.  This, of course, sparked my curiosity, so I did some additional research.

We even had some conspiracy theorists like figgle that went as far as saying "If I was a bit more paranoid I might think that Microsoft deliberately de-optimized their 'open format' file loaders."  The only problem with that theory is that Microsoft doesn't even support Open Office file formats, and there would be no reason to deliberately reduce the speed of its own XML document because Microsoft is committing to XML in the next release of Microsoft Office.  I guess figgle has never tried doing the same thing with Open Office Calc because he would really start breaking out the conspiracy theories.

Others like George Jay resorted to the knee jerk reaction that "Perhaps we should avoid using Windows because it too is bloated".  Not only is it not true, my Windows XP machine boots up in 30 seconds from a cold start to post login, and Microsoft Excel loads in under 2 seconds the first time without the office pre-loader cheat installed.  I can even run Windows XP and Office 2003 on a PC with 64-MBs of RAM.  This is less than some individual Java applications require and any modern Linux distribution takes nearly a full minute to boot on identical hardware using the default installation.  Jay responded by saying that bloat isn't defined by load times and insinuated that Microsoft only loads fast.  The problem with Jay's theory is that Microsoft Office loads fast, reads and writes data fast, and keeps a smaller memory foot print from start to finish.

I like free software as much as the next guy and I had never intended to do a Microsoft Office versus Open Office shootout because it really isn't a fair comparison when development budgets are taken in to account.  Open Office development is supported by donations while Microsoft Office is supported by a massive commercial software development budget.  Open Office is the cream of the crop as far as free software goes, but I couldn't stand idly by while so many people make patently false statements about Microsoft Windows and Office bloat.  I decided to do this shootout of Microsoft Excel versus Open Office Calc on my own desktop PC so that the hardware parameters are identical.  As with any good scientific tests, the methodology must be fully disclosed so that it is repeatable.  You can download this compressed 16-sheet MS-XML file and run the tests on your own.  For Open Office users, download this 16-sheet SXC compressed XML file.

Methodology:

  • Test spreadsheet performance of Microsoft Excel and Open Office Calc on identical hardware with identical 16-sheet sample.
  • Test various spreadsheet data file formats like Microsoft XLS, Microsoft XML, Microsoft Zipped XML, Open Office SXC compressed XML.
  • Record memory utilization of Excel and Calc.
  • Record file creation and file load times.
  • Record file sizes.

Hardware test bed:

  • Intel Pentium 4 3.4 GHz with Hyperthreading enabled
  • FSB 800 with 1 GB DDR 400 MHz RAM
  • NVIDIA Geforce 6800 OC
  • Two SATA Seagate 160 GB 7200 RPM Hard Drive (test files non-OS drive)

Software test bed:

  • Windows XP Professional with SP2 plus all critical updates
  • Microsoft Office 2003 Professional full installation
  • Open Office 1.1.4 Win32 stable custom default installation (Java disabled)
  • Open Office 1.9.125 Win32 full installation

Memory utilization results:

Application and stateMemory KB
Excel minimized to task bar1,304
Excel maximized to desktop7,900
Excel with XML file loaded41,980
Excel after closing XML file7,900
Calc minimized to task bar2,304
Calc maximized to desktop28,484
Calc with native SXC loaded225,872
Calc after closing SXC file55,642
Calc Beta2 minimized4,720
Calc Beta2 maximized38,320
Calc Beta2 w/SXC opened234,784
Calc Beta2 after SXC closed113,680
Calc Beta2 w/ODS opened240,648
Calc Beta2 after ODS closed85,620

File creation time results:

TaskFile creationFile load
Calc creating native SXC98 sec102 sec
Excel creating native XLS1 sec2 sec
Calc exporting to XLS75 sec13 sec
Excel creating Microsoft XML10 sec36 sec
MS XML ZIP compression5 sec9 sec
Compressed MS XML total time15 sec45 sec
Calc Beta2 with SXC169 sec162 sec
Calc Beta2 with ODS112 sec138 sec

File size results:

File typeSize KB
Calc SXC3,683
Excel XLS50,955
Calc XLS62,823
Excel XML192,892
Excel zipped XML3,283
Calc Beta2 ODS3,952

Conclusion:
XML documents are not only slow with Microsoft Excel, but even slower with Open Office compressed XML documents.  Even combining Excel XML creation and ZIP compression times, Excel was substantially faster than Open Office Calc.  Here is a break down of the results:

TaskCreation speedLoad speed
Excel creating native XLSbaselinebaseline
Calc creating native SXC98 x slower51 x slower
Calc exporting to XLS75 x slower6.5 x slower
Excel creating Microsoft XML10 x slower18 x slower
MS XML ZIP compression5 x slower4.5 x slower
Compressed MS XML total time15 x slower22.5 x slower
Calc Beta2 creating SXC169 x slower81 x slower
Calc Beta2 creating native ODS112 x slower69 x slower

As nice as Open Office Calc is for a free application, it doesn't even come close to the performance and usability of Microsoft Office Excel.  There is no shame in losing to a commercially funded application, but no one should make false statements that Microsoft Office is bloated and inferior.  The only remaining question for businesses and homes is whether Excel is worth the money and most have apparently decided yes.

It is well known even by XML proponents that processing XML files are slow.  When a certain technology scales poorly and runs slow on the desktop, it also scales poorly and runs slow on a server.  When software is slowed down by more than an order of magnitude, it is clear that XML will always knock application performance back 10 years no matter how fast our hardware gets.  Most computer users would pay a huge premium for hardware that is 2 times faster, so why would they want software that was up to 98 times slower?  The next time your computer is crawling along, just blame the XML bloat monster.

Newsletters

You have been successfully signed up. To sign up for more newsletters or to manage your account, visit the Newsletter Subscription Center.
See All
See All