Event processing means more than 'speeding up' existing systems

Complex event processing -- now made possible by service-oriented architecture principles -- represents the next stage of business intelligence. However, much work needs to be done to reach this capability.

Complex event processing requires a different mindset and skills

A ebizQ's latest SOA in Action conference, I had the opportunity to moderate a session with Gartner's Roy Schulte, CalTech's Dr. Mani Chandy (CalTech), and IBM's Frank Chisolm in an informative discussion about applying event processing as a strategy for businesses seeking to remain competitive in the years ahead.

However, Roy cautioned, event processing capabilities don't just automatically pop up, even among companies with the most advanced BI infratstructures. "The way you get your systems to be more smart fast and agile is by having the systems designed correctly, and in most cases that means more use of the event processing design methodology," he says.  "You can't just take a conventionally designed system and just speed it up to accomplish the goals that people want to do."

While the technology now exists to build CEP, the methodology requires a different mindset among companies. "The limitation that we have today is that there are not enough people around who understand how to design systems that operate in this fashion," Roy says. "They don't understand continuoius intelligence or complex event processing."

Complex event processing requires continuous streams of information from multiple sources. The good news is that CEP need not be so complex, and, in fact, over the next few years, systems that sense and respond to events will be as commonplace as business intelligence systems are today.

Mani, considered one of the early visionaries of complex event processing, said the "PC-cubed" formula (three Ps and three Cs) will drive CEP forward over the next few years:

• Price – The price of managing data sources will continue to drop.
• Pervasiveness – Sensors, such as mobile phones, have become pervasive.
• Performance – "Enterprises have access to immense computing power that can be harnessed through event processing," Mani says. And now, "parallel, distributed, and cloud computing create ideal environments for event processing."

• Celerity - "Businesses and consumers demand swift action," Mani points out. "You expect to be notified immediately if your plane is late."
• Connectedness – The world is more interconnected. Your company may need to respond immediately to an earthquake in China, a flood in India. Event processing applications help detect events all over the globe."
• Complexity – "Businesses have become more complex, and expect IT to help with increasingly complex problems."

As if laying out the case for complex event processing as "PC" doesn't clarify enough, Mani also explained how a mnemonic -- A, E, I, O, U (but not sometimes Y) -- describes the CEP phenomenon:

• A -- Adaptability: "The event pattern has two advantages, one is loose coupling for application integration, and the other is sense and response," Mani said. "App integration because producers and consumers are coupled in a loose way without knowing about each other. Its easy to add or change the producers and consumers of a system. With the sense and respond aspect, an example is scheduling railroad crews -- a complex problem, a sense-and-response problem.  Because unscheduled events happen all the time, smart railroads are using event processing to adapt."
• E  -- Exceptions: "Computers have to analyze torrents of data to extract nuggets," said Mani. "These nuggets are the events that require a response. A characteristic of smart people and smart systems is that they mange by exception.. they perform continuing operations effectively, bit they continue to detect and respond exceptional situations. Event processing helps separate the critical from non-critical."
• I  -- Instrimentation: "Successful businesses manage exceptional events successfully," according to Mani. "Event processing is used to instrument and monitor the exception and the normal. You will see a rapid rise in business instrumentation and event processing for to improvement of business activity in the next decade."
• O -- Outside: "1960s-90s enterprise IT dealt with mainly IT inside the enterprise. Now the enterprise is responding the events externally," said Mani. "The enterprise monitors actions by the government, its competitors, its suppliers, and its best customers.  The ability to sense and respond to events out side the enterprise using event processing is a significant competitive advantage."
• U -- Unanticipated events: "Enterprises develop event process applications to handle certain types of that they expect, and must also deal with conditions that they don't expect," Mani explained. "Any significant deviations are detected by an event processing application which then sends information about this deviation to appropriate people before the analysis."

Dr. Mani Chandy and Roy Schulte have just puiblished a new book on the subject, entitled "Event Processing - Designing IT Systems for Agile Companies."