6 trends in business intelligence

Once the province of a chosen few, the shadowy, sometimes arcane world of business intelligence, or BI, is now appearing front and center and being deployed to the desktops of the masses.

And for good reason. Customers are using BI tools and analytical software to shorten development cycles, flatten business hierarchies, improve customer retention, overhaul supply chains and strengthen decision-support systems.

We interviewed numerous market experts to pinpoint six key BI trends. Analyze the clues and the opportunities outlined below, and you may succeed in the BI market long before your competitors join the data chase.

Trend 1: More data in more places

Yes, customer data is increasing exponentially. Brian Brinkman, a senior product manager at MicroStrategy, sees some customers whose data doubles yearly.

In fact, we'll generate more information over the next three years than was created in the previous 300,000 years, according to the school of information management and systems at the University of California, Berkeley.

As the data piles up, the hurdles to business analysis rise even higher. Before you can analyze the data, you've got to find it and make sure it's valid--or help your customer do it for themselves. "The wider business community has the same problem that analysts have had for 20 to 30 years--getting ahold of the data and getting it into a format that's useable," notes Mark Battaglia, president of SPSS BI.

Mike Hoskins, president of Data Junction Corp., has a slightly more colorful way of describing the problem. "Data is unruly. Data is never in the format or form that the project architect requires, that the business-intelligence application requires, that the e-marketplace requires. Furthermore, data fights being captured, tamed and corralled. And every time you think you have it penned, a new format, or application, or schema, or business requirement bullies its way on the scene."

What it all means
To get started in BI, get familiar with extract, transformation and load tools. Be sure to approach problems from multiple angles--perhaps a supplier couldn't get a crucial part shipped in time, which means that your client couldn't ship the final assembly. Think about looking for supporting data (in the form of transaction logs or other less-structured formats) from different systems in a variety of locations.

Several factors are driving the BI stampede to the Web, including ease of use and cross-company data use.

Simply put, the focus is all about making it easy to disseminate and use the information across multiple companies, partners, suppliers, and so on.

To enable BI use across the enterprise, "the information presented needs to be personalized to the roles and responsibilities in a business process," says Neil Patil, director of product management at Brio Technology, a BI software developer in Santa Clara, Calif.

Moreover, many potential users of analytical software are not technically astute, notes Clay Young, VP of marketing at ProClarity (formerly Knosys): "Unfortunately, not everyone has analytic training."

Eric Rogge, director of product management at WhiteLight, another analytic software provider, sees his software going into mainstream user environments. "Data display, viewing, interactive display, as well as the addition of your own rules and subsequent recalculation are becoming more necessary," he says.

Jagdish Mirani, senior director of marketing at Oracle, says nontechnical people do indeed need to use analytical software. "Render it all in a portal," he says. "It's a point-and-click world."

What it all means
Get familiar with Web communication protocols and display standards. Bone up on XML skills, because XML will assume an ultra- important role as data is exchanged among partners. Investigate Web portal technology as a delivery mechanism for getting just the right information, in the right amount, to the right people. Understand that not everyone is going to be at the same level, analytically speaking. Structure your offerings and services accordingly.

It's no secret that companies are trying to do more work with less people in less time than ever before. One way to deal with this is to "manage by exception."

Once the key performance indicators for a business process have been defined (for instance, quarterly sales in North America), it's a relatively easy job to monitor those indicators and compare them to historical results. If a predefined threshold in variance is reached, the alarm is sounded and notifications are sent. Opportunities abound in helping businesses define just exactly what those indicators are. And that is where domain expertise in a particular industry or vertical market comes into play.

"You can't prepackage domain experience," claims Anne Milley, manager of analytical strategy at SAS Institute. She maintains that partnering is the only way to help businesses ask the right questions using the right tools.

"Most people manage to five metrics or so," says Ted Stavropoulos, president of FRx Software. "People have their jobs; they don't have time to learn a new tool. Tools are generic and don't deliver any semantic knowledge about the data itself. It's then up to the user to figure out what's important--and what to use."

Brinkman sees activity monitoring, exception-based reporting and notification of events all on the upswing.

Meanwhile, traditional BI is evolving. Reporting may be the big event today, but tomorrow will be a different ball game. Think collaborative-processes predictive capabilities, such as a monitoring interface (for fax, e-mail and pager alerting) and a planning interface (for collaborative work, such as budgeting and forecasting).

Another key trend is that of collaboration. Oracle's Mirani sees BI becoming more collaborative in nature, with some work-flow mechanisms becoming essential as cross-departmental teams work on a particular task.

What it all means
Develop domain experience in a particular area or partner with someone who has the requisite experience. Learn how to create key performance indicators and make yours the best in the industry. Understand what work-flow tools can--and cannot--do.

At a very high level, there may be a number of different analytical systems within a particular enterprise, each of them focused upon a particular line of business or business function. Take, for example, the analytical functionality appearing in supply chain and customer relationship management applications. While each of these "silos" or "smokestacks" may meet the particular business need, they don't give an overall picture of the entire business, at least, not without a lot of customization.

"Applications are being built with analytical components embedded in them. It's good when they work, and bad when they don't," says SAS' Milley. "You need to be sure that the algorithm making the analysis is well thought out and that you understand and agree with what's going on behind the scenes," she adds.

There are those who question the overall value of such application add-ins, because the application is only analyzing the data that it maintains.

Of independent silos, Mirani claims, "Nobody [outside of the system in question] has the same view. [You need to] consolidate at the data layer and give everyone the same view. Yes, it's a fundamental architecture change. Silos are justifiable tactically, but not strategically."

Len D'Amico, VP product marketing at Alphablox, puts it more bluntly. "The growth in packaged applications with analytic capabilities is encouraging, but it doesn't tell you the whole story. There's no view across the business. It doesn't allow insight into the broader problem."

What it all means
Understand the strengths and vulnerabilities of your prepackaged competition. Determine how the overall business can utilize the information being generated from those independent systems.

For years, the cycle of data analysis has not been entirely automated. These days, you can take the functions of data discovery, transformation, loading into a data warehouse, the subsequent analysis and predictive capability for granted. However, taking action on the predictions has been a manual process. With the advent of e- commerce, the opportunity to complete the loop--to act upon the prediction automatically--is viewed by a number of vendors in the BI arena as a huge market in the future.

Whether you're trying to maximize passenger revenue on a particular flight or optimize a supply chain for a particular part, the benefits of--to quote SPSS' Brinkman--"turning insight into action" cannot be understated.

Oracle's Merani is bullish on the concept. He maintains that one needs to "operationalize the data that's been analyzed. Use it on a daily basis; do it on an online basis. Do it in sales, do it in manufacturing. Build the model off-line when and where necessary, but deploy it online. You need to automate the feedback mechanism."

What it all means
Show customers how to increase the ROI quickly on their massive expenditures in enterprise resource planning, customer relationship and e-commerce systems by using the results of analytics as input to these systems.

A great deal of work has been done to integrate more functionality into tools and applications. One argument for that thrust is usability. The number of different tools necessary to "get the answer" can be mind-numbing.

Oracle's Merani claims, "Where you once needed a dozen tools to perform the analysis, now you need just two--the database and the application server."

While Oracle has been waging a war against complexity for some time, it can be a compelling argument when a line of business managers "want ROI now, not two years from now," according to Merani.

But an all-in-one tool has its drawbacks, too. Issues of feature depth and breadth come into play here, not to mention the fact that by going with a single vendor, you're effectively at its mercy. While the job becomes simpler due to improved integration, there's always going to be disparate data from other sources that will need to be included in the analysis.

There's also the issue of the extensibility of applications with embedded analytic capabilities built-in. Is it useful if it doesn't grow with you? SAS' Milley asks, "What's the ROI on a nonextensible system?"

As ERP, CRM and database vendors integrate more capability into the products, the role of the systems integrator may change as there is less work necessary to deliver the required functionality.

"In the future, the system integrator will need to address business issues in the form of domain expertise. A lot of the traditional systems integration tasks are going away," according to Merani.

What it all means
Determine how to augment these packages with your own best-practices assembly of analytics tools and business acumen. No matter where in the cycle you choose to play--whether it's identifying the processes that need measurement, determining metrics, cleansing the data, architecting the data warehouse, using domain expertise to place the right analytics tools which ask the right questions in the hands of the right people, or doing the back-end integration that turns the analytic results into action, there are opportunities at every turn.