Windows 7 memory usage: What's the best way to measure?

Windows memory management is rocket science. And don't believe anyone who tells you otherwise.

Since Windows 7 was released last October I've read lots of articles about the right and wrong way to measure and manage the physical memory on your system. Much of it is well-meaning but just wrong.

It doesn't help that the topic is filled with jargon and technical terminology that you literally need a CS degree to understand. Even worse, web searches turn up mountains of misinformation, some of it on Microsoft's own web sites. And then there's the fact that Windows memory management has evolved, radically, over the past decade. Someone who became an expert on measuring memory usage using Windows 2000 might have been able to muddle through with Windows XP, but he would be completely flummoxed by the changes that began in Windows Vista (and its counterpart, Windows Server 2008) and have continued in Windows 7 (and its counterpart, Windows Server 2008 R2).

To help cut through the confusion, I've taken a careful look at memory usage on a handful of Windows 7 systems here, with installed RAM ranging from 1 GB to 10 GB. The behavior in all cases is strikingly similar and consistent, although you can get a misleading picture depending on which of three built-in performance monitoring tools you use. What helped me understand exactly what was going on with Windows 7 and RAM was to arrange all three of these tools side by side and then begin watching how each one responded as I increased and decreased the workload on the system.

To see all three memory-monitoring tools at work, be sure to step through the screen shot gallery I created here: How to measure Windows 7 memory usage.

Here are the three tools I used:

Task Manager You can open Task Manager by pressing Ctrl+Shift+Esc (or press Ctrl+Alt+Delete, then click Start Task Manager). For someone who learned how to read memory usage in Windows XP, the Performance tab will be familiar, but the data is presented very differently. The most important values to look at are under the Physical Memory heading, where Total tells you how much physical memory is installed (minus any memory in use by the BIOS or devices) and Available tells you how much memory you can immediately use for a new process.

Performance Monitor This is the old-school Windows geek's favorite tool. (One big advantage it has over the others is that you can save To run it, click start, type perfmon, and press Enter. To use it, you must create a custom layout by adding "counters" that track resource usage over time. The number of available counters, broken into more than 100 separate categories, is enormous; in Windows 7 you can choose from more than 35 counters under the Memory heading alone, measuring things like Transition Pages RePurposed/sec. For this exercise, I configured Perfmon to show Committed Bytes and Available Bytes. The latter is the same as the Available figure in Task Manager. I'll discuss Committed Bytes in more detail later.

Resource Monitor The easy way to open this tool is by clicking the button at the bottom of the Performance tab in Task Manager. Resource Manager was introduced in Windows Vista, but it has been completely overhauled for Windows 7 and displays an impressive amount of data, drawn from the exact same counters as Perfmon without requiring you to customize anything. The Memory tab shows how your memory is being used, with detailed information for each process and a colorful Physical Memory bar graph to show exactly what's happening with your memory. I believe this is by far the best tool for understanding at a glance where your memory is being used.

You can go through the entire gallery to see exactly how each tool works. I ran these tests on a local virtual machine, using 1 GB of RAM as a worst-case scenario. If you have more RAM than that, the basic principles will be the same, but you'll probably see more Available memory under normal usage scenarios. As you'll see in the gallery, I went from an idle system to one running a dozen or so processes, then added in some intensive file operations, a software installation, and some brand-new processes before shutting everything down and going back to an idle system.

Even on a system with only 1 GB of RAM, I found it difficult to exhaust all physical memory. At one point I had 13 browser tabs open, including one playing a long Flash video clip); at the same time I had opened a 1000-page PDF file in Acrobat Reader and a 30-page graphically intense document in Word 2010, plus Outlook 2010 downloading mail from my Exchange account, a few open Explorer windows, and a handful of background utilities running. And, of course, three memory monitoring tools. Even with that workload, I still had roughly 10% of physical RAM available.

So why do people get confused over memory usage? One of the biggest sources of confusion, in my experience, is the whole concept of virtual memory compared to physical memory. Windows organizes memory, physical and virtual, into pages. Each page is a fixed size (typically 4 KB on a Windows system). To make things more confusing, there's also a page file (sometimes referred to as a paging file). Many Windows users still think of this as a swap file, a bit of disk storage that is only called into play when you absolutely run out of physical RAM. In modern versions of Windows, that is no longer the case. The most important thing to realize is that physical memory and the page file added together equal the commit limit, which is the total amount of virtual memory that all processes can reserve and commit. You can learn more about virtual memory and page files by reading Mark Russinovich's excellent article Pushing the Limits of Windows: Virtual Memory.

As I was researching this post, I found a number of articles at Microsoft.com written around the time Windows 2000 and Windows XP were released. Many of them talk about using the Committed Bytes counter in Perfmon to keep an eye on memory usage. (In Windows 7, you can still do that, as I've done in the gallery here.) The trouble is, Committed Bytes has only the most casual relationship to actual usage of the physical memory in your PC. As Microsoft developer Brandon Paddock noted in his blog recently, the Committed Bytes counter represents:

The total amount of virtual memory which Windows has promised could be backed by either physical memory or the page file.

An important word there is "could." Windows establishes a "commit limit" based on your available physical memory and page file size(s). When a section of virtual memory is marked as "commit" - Windows counts it against that commit limit regardless of whether it's actually being used.

On a typical Windows 7 system, the amount of memory represented by the Committed Bytes counter is often well in excess of the actual installed RAM, but that shouldn't have an effect on performance. In the scenarios I demonstrate here, with roughly 1 GB of physical RAM available, the Committed Bytes counter never dropped below about 650 MB, even though physical RAM in use was as low as 283 MB at one point. And ironically, on the one occasion when Windows legitimately used almost all available physical RAM, using a little more than 950 MB of the 1023 MB available, the Committed Bytes counter remained at only 832 MB.

So why is watching Committed Bytes important? You want to make sure that the amount of committed bytes never exceeds the commit limit. If that happens regularly, you need either a bigger page file, more physical memory, or both.

Watching the color-coded Physical Memory bar graph on the Memory tab of Resource Monitor is by far the best way to see exactly what Windows 7 is up to at any given time. Here, from left to right, is what you'll see:

Hardware Reserved (gray) This is physical memory that is set aside by the BIOS and other hardware drivers (especially graphics adapters). This memory cannot be used for processes or system functions.

In Use (green) The memory shown here is in active use by the Windows kernel, by running processes, or by device drivers. This is the number that matters above all others. If you consistently find this green bar filling the entire length of the graph, you're trying to push your physical RAM beyond its capacity.

Modified (orange) This represents pages of memory that can be used by other programs but would have to be written to the page file before they can be reused.

Standby (blue) Windows 7 tries as hard as it can to keep this cache of memory as full as possible. In XP and earlier, the Standby list was basically a dumb first-in, first-out cache. Beginning with Windows Vista and continuing with Windows 7, the memory manager is much smarter about the Standby list, prioritizing every page on a scale of 0 to 7 and reusing low-priority pages ahead of high-priority ones. (Another Russinovich article, Inside the Windows Vista Kernel: Part 2, explains this well. Look for the "Memory Priorities" section.) If you start a new process that needs memory, the lowest-priority pages on this list are discarded and made available to the new process.

Free (light blue) As you'll see if you step through the entire gallery, Windows tries its very best to avoid leaving any memory at all free. If you find yourself with a big enough chunk of memory here, you can bet that Windows will do its best to fill itby copying data from the disk and adding the new pages to the Standby list, based primarily on its SuperFetch measurements. As Russinovich notes, this is done at a rate of a few pages per second with Very Low priority I/Os, so it shouldn't interfere with performance.

In short, Windows 7 (unlike XP and earlier Windows versions) goes by the philosophy that empty RAM is wasted RAM and tries to keep it as full as possible, without impacting performance.

Questions? Comments? Leave them in the Talkback section and I'll answer them in a follow-up post or two.