Olympian server performance

System performance is always a complex topic, but never more so than when it involves the Internet.

When I first read Zeus Technology's claims for the performance of their Zeus Web Server on HP's new Itanium 2 Web servers, my first thought was that the CPU is surely a relatively minor part of a Web server's performance. Surely most Web servers are I/O-bound, meaning that even with an infinitely fast CPU the data will have to wait just as long to move to and from the Internet.

But CPUs do matter, at least on an SSL server, which seems to be the point of Zeus' benchmarks. The CPU load of an SSL page is an order of magnitude higher than that of a static, non-encrypted page. The wide instruction design of the Itanium 2 is well-suited to the integer mathematics of the RSA encryption/decryption code in SSL, and the Web servers tested on Itanium use hand-optimized encryption code. The tests are designed to showcase the impact of the CPU.

The tests show huge performance advantages for Zeus and the Itanium 2 compared to the alternatives, which in this case are an HP-optimized Itanium version of Apache 2.0, and Zeus running on Red Hat 7.2 on a 1.3GHz Pentium III as well as on Solaris 8 on a Sun Fire 280R (with a 900MHz UltraSparc III). Are these "equivalent platforms" as Zeus and HP claim? That's hard to say, but the difference is enormous: 751 SSL operations per second on the HP rx2600 running HP-UX with a single Itanium CPU versus 60 on the Sun 280R. If you've got an SSL site (especially an e-commerce site), using CPUs like this to run Zeus would appear to increase your headroom. (Of course, Zeus is Unix-only, and these tests don't include Microsoft's IIS--so for all we know, an Itanium version of IIS running on an Itanium (not a shipping version of it) would outperform Zeus. Someday we'll know.

The numbers above were generated using a very simple HP-developed benchmark called SSLPerf. The benchmark is open source, but the code is remarkable more for its minimal footprint than its open code. And the benchmark is defined more by its testing procedures than the actual benchmark code which appears to be composed of shell and Perl scripts. (Incidentally, in the context of this benchmark an "operation" involves a client retrieving a file containing pseudo-random text using the open source CURL tool.)

: But I think the interesting parts of the benchmark have less to do with SSL than with the programming design of the operating system. Since Zeus is Unix-only, it is usually compared to Apache, the overwhelmingly dominant Web server for Unix. But Zeus has always had a reputation for better performance.

Apache's design, at least until recently, did not lend itself to high performance. The Apache 1.x model--still the default in Apache 2.0--forks a new process for each new client connection. This process is easy and clear to program, but it consumes a relatively high level of resources (data the CPU must manage).

Apache 2.0 adds a new multithreaded model, but takes a fairly simple approach. Instead of one process per connection, now Apache supports one thread per connection. (The various Apache configuration settings that affect these aspects of performance can be found here.)

Zeus took a different approach, one which explains the large performance difference on the same operating system and the same hardware compared to HP's hand-optimized Apache. The SSL portions of each version's code--the heart of the program's critical path--were identical.. Yet when the SSL code was taken out of the picture, Zeus had more than twice the throughput of Apache 2.0 on an otherwise identical system.

The difference is that Zeus implemented what the company calls a "non-blocking I/O multiplexer," which involves a state machine and a small pool of threads, the overall goal of which is to keep operations moving as quickly as possible using the smallest amount of CPU resources.

Ironically, the "non-blocking I/O multiplexer" model is basically the same as "I/O completion ports," which have been in Windows NT and Windows 2000 Server for about 5 years. They're the basis for Microsoft's Internet Information Server, and the reason why IIS usually slaughters the competition in performance tests. From what I can tell from this page on the Apache site, the process model for Apache 2.0 on Windows does not use I/O completion ports, but I'm not sure.

One reason why the Zeus/IIS model is superior for heavy Web server loads is that the world is filled with slow clients and slow links. With dial-up clients, an HTTP request to the server could have a long life-span, perhaps an average of 10 seconds. If each request results in the allocation of a process or even a thread, and that allocation lasts for the lifespan of the connection, the server will hit its maximum load quickly during a burst of traffic. The non-blocking I/O multiplexer minimizes the amount of state allocated on the server for each connection. This is something to keep in mind when you see benchmark results. Do they test the actual conditions, including slow links and low-end clients, or did they use fast clients on a LAN?

Does any of this really matter? Once again, if Web servers are I/O-bound, the performance differences described above will amount to little in the real world. Your own administrators are probably aware of the real issues for their servers. But if you're concerned about performance for your Web clients, it's worth looking beyond Apache.

Does your company use Zeus Web Server? Share your experience in our TalkBack forum, or send e-mail to Larry.