Middleware: Computing's unsung hero

Experts claim the MQ Series is one of the 'top technologies of the computer age'
Written by Steve Ranger, Global News Director

A technical achievement comparable to the "creation of the internet and the PC" or even "the Tower of Babel in reverse". Not the kind of hyperbole you'd normally associate with the glamour-challenged world of middleware but in the case of IBM's MQ WebSphere it may well be warranted.

Further proof of the software's importance to the current computing landscape came at the end of January when the middleware platform become the first piece of software to warrant its own exhibit at London's Science Museum.

The new display (pictured here), unveiled last week, features a giant CD to celebrate the work of engineers at IBM's Hursley Laboratory near Winchester, who developed the WebSphere MQ -- message/queue -- software.

In June last year the IBM engineers won the Royal Academy of Engineering's MacRobert Award -- and a cool £50,000 to share between them -- for their work on Websphere MQ. Previous engineering marvels to win the prestigious prize include the Harrier Jump-Jet engine and the roof structure of the Millennium Dome.
Dr Tilly Blyth, curator of computing and information at the Science Museum says that while the museum has displayed digital art and technology before, this is the first time it has installed an exhibit purely about software.

She says the MacRobert award helped to highlight a shift in the way that computer science is currently perceived by the scientific establishment and the general public. "We can no longer afford to represent computing as the pursuit of lone individuals. With the spread of networks and applications working across them we can now see computing as a truly social device, enabling the proliferation of services and communication across multiple platforms."

Launched in 1994, WebSphere MQ is now used by over 10,000 customers, including more than 80 per cent of companies in the Fortune 100. Primarily it allows systems to exchange messages even though they may have different, and potentially incompatible, operating systems. Formerly known as MQ Series, the software has been in development for over 10 years and provides automatic application integration between multiple platforms without the need for custom coding

There are two basic components to 'message oriented' middleware: the message and the queue. The messages can range in size from millions of bytes worth of data down to just one or two bytes, transmitting data such as order records or customer payment details. But each message has to be sent only once -- or this could lead to problems such as duplicated orders, which then have to be cancelled.

Using MQ these messages are put into a mailbox known as a queue, and the message then stays there until it is needed, so that users can get access to the business information when they're ready to, and in the order they want it. Not exactly revolutionary, you might think -- but it was a major shift in the way systems communicate, and a massive engineering project.

Tony Storey, IBM Fellow, and one of the original inventors of WebSphere MQ, says work started on the software around 1987, when companies in the financial sector began to run into a new kind of problem with their IT systems.

"A lot of companies had installed different kinds of systems that ran on different operating systems and different protocols and it was incredibly difficult and costly to integrate them," he explains.

"We tried to define a set of standard APIs and a standard protocol to run on all the systems to link them together. The more difficult task was trying to provide an implementation that ran on all these different platforms. This was a big software engineering challenge," Storey says. And in 1998 IBM added the MQ message broker, moving the middleware from a point to point, to a hub and spoke architecture, extending its use further.

"We live in this incredibly networked world and there are so many pervasive devices out there and this is the technology that ties all of this together," Storey says.

Dr Robin Paul, chairman of the Royal Academy of Engineering's MacRobert judging committee says awarding the prize to Websphere MQ has extended the idea of what engineering is about: "It is an undoubtedly brilliant innovation which has conquered the world."

He sees Websphere MQ as "the Tower of Babel in reverse" by allowing different systems to communicate. Another important element for the judges was reliability: "The other impressive thing is that every message is sent once and only once - you don't want a cheque for a million pounds getting cashed twice."

Mike Thompson, principal research analyst at the Butler Group says Websphere MQ does deserve this kind of respect, and is one of the most important developments in computing, along with the Internet, the PC and the relational database.

"The great thing about it is that it is asynchronous, which is the key. It will queue the messages until the other application becomes available - and with the queue comes guaranteed delivery. It's probably one of the top technologies of the computer age," says Thompson. "A large part of the banking system relies on MQ Series because it's a proven platform. When an application sends off a message it will be delivered."

Neil Ward-Dutton, director of technology practices at analyst Ovum says WebSphere MQ has been very influential -- especially in terms of how big organisations put together heavy-duty transactional systems.

"Without something like MQ you can send a request from one system to another, but the fundamental thing that is missing is reliability. [Without MQ] what happens is that if the network goes down the message is lost. What MQ does by using the idea of queuing is it provides a very resilient system for sending messages," he says. "One of the things that gives you is very high levels of predictability and scalability, so systems can react in a much more graceful way. All the really heavy duty transaction systems run it so it's really part of the furniture."
And IBM is still finding new uses for MQ technology, particularly in terms of pervasive computing, and collecting data from small devices such as temperature probes or radio frequency identity tags.

It is, for example, helping chemists at the University of Southampton remotely monitor experiments from smartphones. Sensors in the lab gather data on temperature, light levels and motion in the room and relay this to the MQ message broker, which makes the data available to anyone authorised to see it via the web or a smartphone using GPRS.

In the next phase it will be possible to use a Web page or on a smartphone to turn something on or off in the lab, explains IBM's manager of pervasive messaging technologies, Dr Andy Stanford-Clark. WebSphere MQ might have its own museum exhibit, it is clearly no museum piece yet.

Editorial standards