Mavericks memory compression review

Mavericks memory compression review

Summary: Compressed memory is the only major storage feature in OS 10.9 Mavericks. Here's what it is and how well it works.

SHARE:
TOPICS: Storage, Apple
21

For most of Mac history using the standard memory configuration meant a world of hurt. The machine would boot and work well - usually - with one app at a time, but open a few Safari tabs, Mail, a media player and a word processor and switching apps gets s-l-o-w.

In a virtual memory OS - all consumer/server OSs today - physical memory is extended by using mass storage - disk or SSD - to store inactive or little used memory pages. That frees physical memory - DRAM - for use by active programs.

The downside is that moving those pages back to DRAM takes a disk several hundred thousand times longer than accessing them in DRAM. An SSD takes thousands of times as long.

Compressed memory
A new feature of Mac OS 10.9 - Mavericks - compressed memory, increases the effective size of DRAM through inline data compression. This isn't a new idea: over 20 years ago the HP Omnibook 300 used inline compression to double the effective size of its 10MB compact flash card.

What is new is that with multiple cores running an optimized compression algorithm the system can compress/decompress data much faster than swapping to disk or SSD. This saves time and energy, since the system isn't idling waiting for memory page swaps - important for notebooks.

And there's nothing to configure: it works automatically in the background. All you see is a more stable Mac with more memory.

What this means
My main machine is a 2012 MacBook Air with dual-core 2GHz i7, 500GB SSD and 8GB RAM. It is as fast as my 2006 quad-core Xeon Mac Pro was and a lot easier to carry.

But under Mac OS X the swap file would grow quickly as I loaded apps - and especially Safari tabs - and once the swap file reached about 3GB the pain of an app switch would slow down the machine. The fix: a restart with a shutdown that takes longer than the bootup. Clumsy.

Due to Safari memory leaks the swap file could grow to over 16GB. At that point the machine was unusable: drop down menus would take 30 seconds to appear; app switches even longer; and a very long shutdown.

The free utility Memory Clean helped by freeing up physical DRAM and reducing restarts. But MC needs to be manually invoked and takes about 25 seconds to run on my MacBook Air. Faster than a restart but more hassle than I like.

What's different
With Mavericks memory compression I'm seeing much improved behavior from my MacBook Air: tiny swap file sizes; greater stability; and no performance hit. Even with Safari open for almost 4 days and dozens of open tabs, Final Cut Pro, Preview, Mail, VLC, an FTP client, text editor and the usual dozens of background tasks the swap file is a tiny 114MB where normally it would be at least 2500MB - hurting performance - and growing fast.

It is the single biggest improvement so far in my Mac thanks to Mavericks. You can see for yourself in the new Memory pane in the Activity Monitor.

The Storage Bits take
If you were skeptical about the value of a free OS update, fret no more: this single feature makes a lightweight notebook much more viable for power users. As John Siracusa put it in his excellent Mavericks review:

It's a capability win; Mavericks can handle much more demanding workloads than previous versions of OS X before crying uncle.

The ability to easily handle my daily workload is the most important benefit. I'll still use Memory Clean as preventive maintenance, but by effectively doubling the DRAM in my MacBook Air I feel like I've got a new machine.

And if you're buying a new Mac with non-expandable memory, you may not need to buy as much of Apple's high-priced RAM. It's like getting an extra 4-6GB of RAM for free.

Now Apple should buy Disk Warrior and make it a standard OS utility to fix directory cruft!

Comments welcome, of course. What's your favorite Mavericks feature?

Topics: Storage, Apple

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

21 comments
Log in or register to join the discussion
  • Interesting take on this

    However, you have spoken previously in depth about data corruption as it relates to file systems.

    I am curious whether this kind of inline compression increases or decreases the potential for data corruption, and what if - if any - steps are taken to mitigate risks?
    ITTech001
    • Good question!

      The inline data compression doesn't go use HFS+, whose bugginess causes the data corruption problem. The specific algorithm has been around for over 10 years and assuming the code is well optimized and tested - and as a core system function it should be given all the betas - I don't expect to see additional problems.

      Compression is a widely used technology - all tapes use it and, increasingly, new gen storage - so this isn't a big leap technically. Replacing HFS+ is the next big win for Mac storage.

      Robin
      R Harris
      • What data corruption.

        We still have yet to see you actually run a scenario to empirically demonstrate ths corruption at work.
        baggins_z
        • Data corruption

          What is demonstrated, is that HFS+ does not detect data corrupted by media, or by the disk controller etc. Thing is, none of the page based file systems (almost all known file systems around) has any better protection towards this than HFS+.

          The only cure to such problems are storage platforms such as ZFS, where each data block is protected by an cryptographic checksum, created and verified outside of the storage subsystem.

          I happen to keep around an 200GB Seagate drive, that is tested "perfect" under page based file systems, but always fails with bad checksums under ZFS --- that drive is guaranteed to corrupt any data you store on it, that is not cryptographically verified.
          danbi
  • Better to fix Safari memory leaks

    Memory compression is all very good. But the article points out memory leaks in Safari so the tabs over time give rise to problems. It's better to cure the underlying cause than to put on a band-aid.
    Chas Young
  • Haven't looked at Safari specifically, but

    it isn't as badly behaved as it used to be. Maybe sandboxing fixed a bunch of problems? In any case I'm seeing a better Safari experience.

    Robin
    R Harris
  • Seriously?

    If you're opening Safari with multiple tabs and it consumes 8GB of RAM, something is VERY wrong with Safari's memory management.
    Joe_Raby
  • Memory compression won't fix this problem.

    It may delay the decrease in performance but the leak will eventually consume all available memory no matter how it is obtained.
    ye
  • What's old is new again.

    Connectix used to sell Ram Doubler for the Mac way back when. Ah, the good old days of manual memory configuration on the Mac. Still better than IRQ and COM port conflicts!
    ye
    • Lol

      Thinking the same thing. I remember when Ram Doubler + Disk Doubler practically turned my IIsi into a brand new machine!
      dsf3g
      • Ram Doubler + Disk Doubler

        Those were awesome utilities. Once AutoDoubler came in that completed the entire circle.
        TimeForAChangeToBetter
  • Robin is a product whore...

    just loves to suggest his friends products to everyone for purchase. SLUT!
    DontUseMicrosoftAtAll
  • Nice topic, but invalid points.

    Rob, you chose and interesting topic, but I don't agree with some of the claims you're making.

    1. "For most of Mac history using the standard memory configuration meant a world of hurt."

    That's not really any more true in past history than it is today. This is more of a BS comment right up front to try to draw more importance to your story. Yes, eventually, after multiple OS upgrades, upgrades to productivity software (and if we're talking Mac classic, loading up with tons of extensions), it would eventually be the case, just like it was with any other hardware vendor / operating system. However, take the iMac for example. It's come with a minimum of 8GB for some time now. Is 8GB really not enough? Now, yes, if you want to make the point that the very low end if Apple's line such as the Macbook Air and Mac mini devices which only come with 4GB. That would be more of an edge case and not quite the point you were trying to make.

    2. "Due to Safari memory leaks the swap file could grow to over 16GB."....

    Activity monitor is your friend. Unfortunately, you have to be smart enough to understand what you're looking at. The large swap file is a symptom of another problem, not the problem itself. Look at your applications from time to time and learn what "normal" usage is like from time to time, I've seen memory leaks in multiple different programs. Yes, Safari is one of them. iTunes can be another, etc. Someone I know once had a G5 iMac and the Dock would have a huge memory leak. The point here is, you only need to quit the offending program and relaunch that program to return those resources back to normal. Restarting your system is basically an acknowledgement that you don't have the ability to easily identify and correct such situations. It only takes a few seconds at best.

    3. "The free utility Memory Clean helped by freeing up physical DRAM..."

    Yes, Apple also has a free command line utility called "purge" which does this (comes with free developer tools). The problem is, comments like this indicate that you don't understand the difference between "free" and "inactive" memory. BTW, Linux handles memory the same way. Yes, you can purge the inactive memory and thereby showing more free memory, but you can slow down your machine in the process because you're dumping caches and various other types of memory buffers used to make your machine faster. When this space is needed, the OS itself will purge the inactive memory it needs, etc.

    Anyway, the bottom line is this. If you leave programs running for days on end like Safari, iTunes, etc. then eventually, you may run into problems like you describe. A quick look at Activity Monitor should make it clear where your problems are. Most often, simply quitting, then restarting a single program will take care of your issues. If you have to start goofing around by manually purging inactive memory, then you probably need to either change some of your computing habits (like quit unused programs when you're not using them) or buy more memory. Finally, if Maverick's compressed memory feature helps minimize this problem, then we can both agree that's a good thing.
    techconc
    • Are you sure?

      "you only need to quit the offending program and relaunch that program to return those resources back to normal" Although I'm not familiar with the Mac OSX memory model, isn't a memory leak also a piece of un-managed memory that is not freed up when an application is closed?
      jsargent
      • Depends on definition I suppose...

        At the most basic level, a memory leak simply refers to a situation whereby a process allocates memory, but does not free it up when it is no longer needed. This memory is reserved an neither available to the same process nor other processes that need the memory. In every case I've ever seen, it has always been an errant application that keeps allocating but failing to release memory. I've never seen a situation where memory just disappeared and was no longer reachable by any application, even after the application has closed. While that situation may be possible, every instance I've seen and even read about online has been with regard to errant applications whereby you can see the ridiculous memory allocation within the Activity Monitor program. Quitting the application always frees up the memory. At least for this type of memory leak. Again, I've not witnessed any other type on OS X.
        techconc
        • Experience?

          Then you haven't seen a lot of memory leaks. Many programs don't free up the memory as they close, leaving it unavailable until the system is restarted.
          vermonter
  • techconc

    What you say is overall true, but there should not be any applications from Apple that have memory leaks. Safari was terrible from that standpoint, and having to quit applications on a regular basis and restart them is just as primitive as having to reboot the OS on a regular basis to "clean things up".

    As users, we may be smart in figuring out which app has the memory leak, but apps from a major vendor should not have memory leaks. Xcode has plenty of tools for finding memory leaks and I assume Apple knows how to use them.

    Safari is much better behaved now, though. Nothing to do with memory compression, which would only postpone the pain, so somehow Apple fixed the memory leaks directly or as a side effect of sandboxing each tab.

    Again, I agree that a lot of what the author is chalking up to memory compression is not due to memory compression at all. And I agree that turning inactive memory into free memory is a mirage. But Safari seriously needed fixing for a LONG time, and it appears Apple has finally done it.
    1wayne
  • Masking the problem

    1wayne,

    I agree with your points. End users shouldn't have to worry about such things. Apple is doing some of the right things such as ARC for Objective-C programs, etc. However, any libraries that are using C, still use malloc / free that have to be invoked manually. Yes, there are tools such as "leaks", etc. to help developers, but this type of bug is neither uncommon nor unique to OS X, etc.

    I suppose what bothers me most about an article like this is that it's sort of giving praise for having a better ability to mask the same problem. As you know, compressed memory doesn't fix another program's memory leak. It just hides the symptoms longer. What's more, even with memory leaks for application software, the problem is often resolved simply by changing usage habits like quitting applications when you're done with that session. For example, I occasionally see an issue with iTunes, because I leave iTunes running 24x7 as a media server for things like AppleTV, etc. When I see it go up over 1GB, it quit out, restart and all is fine. It might take weeks before I have to do that, but it happens once in a while.

    Anyway, to your point, we shouldn't have to worry about such things. But the reality is, we have no control over such things either. I've never had to reboot because of a low memory situation though. I also know that running utilities to purge inactive memory isn't the solution either. Likewise, the few times I have to deal with such things, I choose a short trip to Activity Monitor, sort by memory usage and recycle the offending app. It's something I rarely have to deal with and system performance never gets anywhere near as bad as Rob makes it sound. Perhaps there's a bit of embellishing going on for dramatic effect. In fact, the uptime on my primary Mac is limited only by the last power failure or the last system update that required a reboot.
    techconc
    • Agreed. The author has unnecessarily embellished system memory issues

      No way will memory issues alone slow down the Mac OS X system in the manner that the author describes. Besides, identifying a runaway application and restarting it is a far superior solution to shutting down all applications and restarting the computer. Clearly, the addition of compressed memory is of benefit to all users, and it will particularly aid users who acquired computers equipped with less than 8 GB of RAM before it was standard due to its high cost and are stymied from augmenting it because Apple soldered the RAM chips to the motherboard, making upgrading memory impossible.
      davidlfoster
  • DiskWarrior

    On the one hand I'd like to see DiskWarrior available on every Mac. It's a wonderful app and routinely finds errors that Disk Utility overlooks - I use it regularly. On the other hand, I'm concerned about what Apple ownership would do to the development of the product. Over the years Apple support for their own software has been spotty at best. And if they included DiskWarrior with OS X (most feasibly in the Recovery HD partition) they would lose the profit incentive that keeps DiskWarrior competitive. If you're not sure what that means, consider how little improvement there has been in Disk Utility over the years. It has gained a feature here and there, most notably the ability to change partition size and add and delete partitions under certain conditions, but on the whole it has remained much the same as it ever was. Most updates are for compatibility with new versions of OS X, but with these it usually looses backwards compatibility with older versions of the operating system. The latest version of DiskWarrior, on the other hand, works on systems as far back as 10.3.9 Panther. I can't imagine Apple doing as well.

    Right now, if a user knows enough to use DiskWarrior, they will usually find a way to own it. In my experience most users don't even know how to use Disk Utility - when and if they know it exists. DiskWarrior would be lost on them. If DiskWarrior were hidden away in the Recovery HD partition - as it would have to be to be used on a single drive system - it would rarely see the light of day. So I, for one, prefer that DiskWarrior remain an independent product.
    thewhitedog