Find out what your server workload really demands

ZDNET Recommends

The best cloud storage services

Free and cheap personal and small business cloud storage services are everywhere. But, which one is best for you? Let's look at the top cloud storage options.

Read now

One of the hardest parts of infrastructure sizing is storage, especially storage performance. That's partly because most I/O tools focus only on IOPS, but also because I/O needs depend on your workload's range of I/O sizes and frequency. No single number tells you what you need to know.

Small I/Os - say 4KB - will typically max out the controller long before they come anywhere near maxing out bandwidth. And large I/Os will saturate bandwidth while the controller is barely breaking a sweat. Since both controllers and bandwidth are expensive, it pays to know how much of each your workloads really need.

At the 2019 Non-Volatile Memory Workshop last week I learned about a free workload profiling service. Back home I tried it, and danged if it didn't show me the unexpected ways Final Cut Pro stressed my quad-core iMac.

Here's how

Go the website testmyworkload.com. They offer three lightweight applets - Windows, Linux, macOS - that captures storage I/O data. I downloaded the macOS version - less than a megabyte - installed it, and started a 10 minute I/O capture.

Then you go to the web site, upload the file the applet creates, and in a few seconds you can look at the I/O activity on any device or array connected to your system or server. You can slice the data by IOPS, size or application.

Learn more about realworld workloads at the Storage Networking Industry Association (SNIA) website. There's a detailed white paper.

Results

I assumed that Final Cut Pro, a video editor, would require lots of large block streaming transfers as I started editing. But it turned out that there were lots of 4KB transfers among the 128KB transfers. The IOProfiler gives a much more detailed look at I/O than I've seen before.

In a 15 minute period, my iMac had over 225,000 I/Os, or 2500 IOPS, quite a bit more than I'd expected. It turns out that there are a lot of system I/Os that inflate the storage numbers.

Why?

I always wonder about what a "free" service expects to get from my participation. According to Eden Kim, the CEO of Calpyso:

TestMyWorkload is an entry level freeware to help evangelize the market and let users become aware of the basics regarding real world workloads.

The Pro version of the TMW software offers a number of advanced features, such as much finer grained data capture and the ability to run it locally with no uploading.

The Storage Bits take

I often observed that customers had little understanding of their actual workload requirements. As a result they could easily be frightened into massive over configuration. With this tool, you can be in the driver's seat, knowing what your application requirements really are, and not be stampeded into an overpriced or configured storage system.

Courteous comments welcome, of course.