Performance analysis of OpenOffice and MS Office

Performance analysis of OpenOffice and MS Office

Summary: Microsoft Office came out very lean and fast while OpenOffice.org Office Suite was just the opposite. Some couldn't accept the numbers and complained that the Task Manager numbers may be inaccurate and hiding memory usage. They demanded more proof, so here it is.

SHARE:
TOPICS: Tech Industry
431

In my last blog, where I did a high-level technical evaluation of Microsoft Office 2003 and OpenOffice.org 2.0, I showed that OpenOffice was a memory and resource hog.  Contrary to popular belief (among Open Source advocates), Microsoft Office came out very lean and fast while OpenOffice.org Office Suite was just the opposite.  Some couldn't accept the numbers and complained that the Task Manager numbers may be inaccurate and hiding memory usage.  They demanded more proof, so here it is.

It doesn't matter how fast the CPU is, OpenOffice is simply bloated.

To get more granular and detailed memory and processor consumption data, I downloaded a copy of Process Explorer from SysInternals and used it to gather a wide range of Data.

Here is a comparison of memory and CPU usage between Microsoft and OpenOffice.org office applications.  This is with just the bare application and blank data file loaded.  Note that SOffice.exe resource utilization was very minimal and is probably just a basic launcher.  To keep things simple, only SOffice.bin was counted against OpenOffice.org.

From this table, we can see that OpenOffice.org is indeed a memory and CPU hog.  It is literally taking up about 10 times the processing time and memory to just load the application itself along with a blank file.  It doesn't matter how fast the CPU is, 10 times is 10 times and OpenOffice is simply bloated.  If that doesn't convince you, look what happens when we try to load our large 16-sheet sample.

Here is a comparison with the standard 16-sheet SXC and XML sample file I've been using.  The sample is in compressed XML format because it is smaller and easier for you to download.  You'll have to convert the XML file to XLS and the SXC file to ODS to run the following test yourself.

Even when dealing with what is essentially the same data, OpenOffice Calc uses up 211 MBs of private unsharable memory while Excel uses up 34 MBs of private unsharable memory.  The fact that OpenOffice.org Calc takes about 100 times the CPU time explains the kind of drastic results we were getting where Excel could open a file in 2 seconds while Calc would take almost 3 minutes.  Most of that massive speed difference is due to XML being very processor intensive, but Microsoft still handles its own XML files about 7 times faster than OpenOffice.org handles OpenDocument ODS format and uses far less memory than OpenOffice.org.

Now to be fair, OpenOffice.org is free and is cross platform, but does this really matter to the 90% of the users in the world who only use Windows?  Does this change the fact that OpenOffice.org is a CPU and Memory hog?  Microsoft Office Professional on the other hand costs about $240 when bundled with hardware or if you look for OEM pricing.  I'll leave it up to you the reader to determine if your money is worth more than your time.

Preview blogs in this series on OpenOffice.org versus MS Office

Topic: Tech Industry

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

431 comments
Log in or register to join the discussion
  • OO Vs Ms Office

    You get what you pay for....
    Scrat
    • Amazing isn't it

      When you get more. It's a hard idea to get used to. Actually, I haven't quite internalized it myself, but real-world results, rather than artificial experiments, are hard to argue with.
      IT_User
  • Securing the Network?

    Hey George, why you so busy beating on OpenOffice? Aren't you supposed to be talking about networks or something?

    Seriously though all these benchmarks and everything with so little information about what OS you're using, the CPU, the RAM, the files, the entire setup.

    We understand that you think OpenOffice is a pig. We also understand that you find absolutely no use whatsoever for XML.

    Now please stop projecting your personal hatreds onto things and either discuss what you're supposed to be discussing or learn how to properly do benchmarks.
    Robert Crocker
    • If the results aren't favorable, they are flawed?

      Why don't you stop attempting to attack George, and instead go demand an answer from the writers of Open Office as to why their product is a piece of crap?

      Just because something is free doesn't mean it has to be garbage.
      Scrat
      • The problem with that is..

        I haven't been able to get the same results that George does. So why would I demand them to speed something up that I don't think is too slow in the first place.
        Patrick Jones
        • Really? When did OO.o refute these results

          Do you know something I don't? When did OO.o refute these results?
          george_ou
          • Questions

            Not everyone reads Zdnet let alone your blog. Besides I think he was actually saying that he has as of yet been unable to recreate the results that you have not that OO.o has refuted your results. That is the point of putting up your test data right? So that we can verify your results?
            For our testings can you provide the number of times you preformed these tests. Did you average the results or the best case? When you repeated the test for loads did you restart the app or leave it open? What utility did you use for measuring the memory and cpu consumption? Over what time interval did you take the readings? Were those readings averaged or best case over the time interval?
            Rokstar83
          • I've notified them, so have others

            The answers to your question is in the blogs. I used Process Explorer. I did not post multiple tests since they varied very little from test to test.
            george_ou
          • I understand...

            I understand that you didn't post multiple tests... I saw that... cause there was only one listing. I was just curious the number of times you tested as well as how you arrived at the numbers you did. Over what time interval did you look at the numbers and by what means did you get those numbers? Was it an average or a random selection? Did you restart each app for each subsequent test? If you did, you may want to see what happens if you didn't and visa-versa. OO.o has to link the dlls on load, something that MSO doesn't need to do.
            My appoligies on the process explorer I must have miss that.
            Rokstar83
          • You misquoted me

            again :)

            "I haven't been able to get the same results that George does."
            Patrick Jones
          • Calm down, George

            Your numbers, and just eyeball evidence indicate that OpenOffice is lagging from a performance perspective. It is likely that those in your position, routinely manipulating spreadsheets that are off the scale of what Outlook terms 'enormous', would find OpenOffice unacceptable in its current state. Unnecesarily large and slow code are not desirable, and certainly these will lead to thresholds where the effect on performance is noticeable.

            The objection is with the extension of your results into the realm of the everyday user. When dealing with ordinary size documents and spreadsheets, OpenOffice is, to all intents and purposes, instantaneous - that is to say, no discernable lag.

            So OpenOffice might be restricted to something like 97% of the population. That's not bad, but certainly worth noting. It's also possible that functionality was the goal of version 2.0 and that attention for 3.0 will be directed toward performance improvements - that's not an unusual version strategy - and might push that threshold to, say, 99.5%. But it might be that MS Office wiould always lead in terms of performance and there would be a segment of the market or which there would be no alternatives.

            It would be far more helpful if, rather than being defensive about your figures, you did some further analysis to find the threshold file size where the current OpenOffice begins to impose intrusive delays. Then it would make sense to advise the reader [i] I'll leave it up to you the reader to determine if your money is worth more than your time.[/i] Since the average user doesn't run into that tradeoff, it just comes across as a gratuitous statement.
            IT_User
          • Very well said...

            IT_User,

            I thoroughly enjoyed your points. I think Ou would look much less "adversarial" if he were to do the analysis and show where OpenOffice clearly starts to separate from MS Office in terms of "acceptable waits". Granted, this does not necessarily play into Ou's on baises and advocacy goals.


            Jim O'Flaherty
            jim_oflaherty
          • I agree

            "The objection is with the extension of your results into the realm of the everyday user."

            I couldn't agree more with you.

            My personal experience is, I haven't used Excel or Calc very much, but I did use, in the past, Word, and now Writer. My experience (and the people I know, namely my girlfriend and college mates) is that Writer has no noticeable lag. I didn't get a cronometer, but I never really noticed.

            And, particularly, Word is just plain unable to handle 50+ pages documents without hanging or doing whatever it desires. I made a few 130+ docs with Writer, and *my experience* was really good (YMMV).

            I don't have the latest hardware, never did (though now I'm quite fine, with my P3/1ghz), so perhaps those hangs would be suppresed if I had used an older (less resource hungry) version of MSO... the thing is, on the same hardware (this machine is almost yrs old, and I tried MSO2k, MSOXO and 2k3, and OOo1.1.4, OOo2beta, and now OOo2 final) Writer performs better.

            Yet I didn't benchmark it...

            One thing I really (REALLY) dislike about Writer is that it's trying so hard to become Word (it seems there's no other way to grab marketshare) that right now it's a pain in the ass if you want to write focusing on content, not presentation (I would like some TeX based thing with a superb UI, just like Word, or Writer, you know, no more Bold, Italics, Colors, or even viewing the document framed in a page, only styles... it would be simpler in a way... perhaps MS could integrate something like this is MSO, something that would prepare a document, and you could "compile" it to a Word document with a nice formatting...).

            That's my experience.

            NachoKB
            nachokb
      • The benchmark

        As highly commendable as George's effort is, measuring memory and CPU usage through the Windows Task Manager is not the most accurate method. It is there for quick inspection reasons, not for debugging or benchmarking.

        Yes, OOo mostly definitely is slower and heavier than MSO, but not in a way that harms my daily work. I use OOo at home and in my laptop, and MSO at the office. Despite the performance issues, OOo gave me a lot of flexibility and actually aided my productivity.

        "Just because something is free doesn't mean it has to be garbage."

        It's not garbage. Just because something is free it doesn't mean you have to treat it like garbage, or make benchmarks to write it off.
        Anti_Zealot
        • My mistake, sorry.

          Just to point out I saw my mistake, he used the Process Explorer, not the Task Manager. My apologies.

          My personal opinion and experience on this are not influenced by the blog, however.
          Anti_Zealot
    • Thought police

      OOO - the thought police strike again. Whatever happened to the idea of free speech?
      sandpiper
  • ODF write times?

    So you going to give us a table on how fast OpenOffice is compared to Office in writing ODF files?
    Robert Crocker
    • I can add that tonight

      I'll update with the create process tonight.
      george_ou
      • ODF format

        I would be interested in the ODF format performance too. Especially since the .ods format will be the one that is the new native spreadsheet format for Openoffice.
        John Simpson
  • Lies, damn lies, and statistics...

    ... and George has managed to cram all three into this.
    Henry Miller