Ever since the early 2000s, when I became aware of the first data mining systems, I have found the technology behind predictive analytics to be compelling, even fun. By building models that track observed patterns in existing data, they permit the prediction of the value of a given field/column based on new data for the rest of the fields in the table or data set.
Back in those days, data mining seemed more powerful than conventional OLAP-based BI technology. In fact, it almost seemed magical. And, as it happened, it was easier to implement in many respects than were systems based on OLAP cubes and drill-down analytics. In 2004, when I assumed a leadership position in a boutique consulting firm focused on BI, I was excited at the business prospect of data mining software and predictive analytics.
But data mining never took off. Despite the business utility promised by a technology that could predict who would be your best customers, could optimize marketing budgets, and even prevent outages or calamitous events through advanced warning, customers didn't take to it. Instead, they wanted to stick with the drill-down analysis they already knew.
Fast forward to the Big Data era, and the technology has come back into the spotlight. Companies like RapidMiner, Revolution Analytics (acquired last year by Microsoft) and others have joined the ranks of veterans like SAS and IBM to break into the predictive analytics field.
Has that made the technology mainstream? While adoption of predictive analytics technology has definitely increased, it still largely occupies the territory of specialists. Programming in R is not straightforward; picking algorithms and tuning their parameters isn't very easy either. So where's the next frontier for predictive analytics, in its quest to become common and widely adopted, by business users as well as specialists?
What's the answer?
For some time now, I have thought that the best way for predictive analytics to become more mainstream is for it to cease being a distinct product category. Rather, it needs to be embedded in broader analytics software suites, and in business user software as well.
One company in the predictive analytics space, BeyondCore, is doing something about this. While the company has for some time had a standalone predictive analytics product, it now also offers BeyondCore Analyst for Office, which integrates into Microsoft Office, including its Web clients.
Demonstrated to me last month by BeyondCore CEO Arijit Sengupta, I saw how this Office add-in filters out "discoveries" that are not statistically valid, and explains its findings, not just by publishing a model, but by producing a visual presentation with explanatory text, inside a Word document or a PowerPoint presentation.
The add-in is available on a freemium basis, permitting 10 analyses per month, with a data maximum of 10 columns and 100,000 rows. The Word and PowerPoint add-ins can connect to a variety of data sources; an additional add-in for Outlook provides consumption-only capability.
Will BeyondCore solve the problem of predictive analytics as a sleeper area of technology? I don't know, but I do think the company is on to something. Predictive analytics shouldn't just be deployed as cloud services for data scientists (as Microsoft's own Azure Machine Learning product is), or as a standalone programmer's tool (as R and R Studio are). It should have a self-service interface, and be available from environments where business users are already doing their work.
Microsoft would do well to promote BeyondCore Analyst for Office. BeyondCore would do well to integrate it with Azure Machine Learning, HDInsight an other elements of the Microsoft Data Platform. Other vendors in the space should take a cue from BeyondCore's lead: make predictive analytics easier, more integrated into common work environments, and more automated.
As with other software categories, reduced friction will likely lead to increased adoption. Increased adoption of predictive analytics would be a great growth accelerator for the business analytics market overall. The industry should make this happen.