Analytics in 2018: AI, IoT and multi-cloud, or bust
In this year's predictions roundup for data and analytics, we find that AI, IoT, and the "multi-cloud" concept are all the rage. But cautions around data governance/regulation and irrational AI exuberance subdue the mood.
At the end of every year, tech PR firms circulate and hawk the prognostications of their client companies' executives on what the next year will bring in the world of data and analytics. There are almost always contradictions to be found on certain points and suspicious unanimity on others. And because the predictions tend to function as self-serving marketing messages, sometimes they can sound more like taglines than substantive forecasts.
That may sound a bit snarky but -- I gotta say -- even if it's a lot of work, it's always fun to read and sort out these predictions. Categorizing and finding some consensus in them can be very useful as, together, they provide important identification of market trends, not just around what customers will need and implement, but also what the vendors themselves will pitch and prescribe.
Big themes, long post This year, most of the predictions addressed the growing importance of the Internet of Things (IoT); machine learning (ML) and artificial intelligence (AI); the emergence of the "multi-cloud" imperative; and the twin issues of data protection regulations and data governance.
Compiling this year's predictions has produced what I must admit is a post rather epic in length. But there's real value in the collective analysis of our motley crew of forecasters. And with me signing off until next year, leaving you with some long-playing content seems, in any case, worthwhile.
The big ticket in predictions this year is definitely around AI and ML, so that's where we'll begin.
Oracle's oracles Redwood City might seem a funny place to start the AI thread, but the fact remains that the folks at Oracle have some serious AI love going on. And starting there certainly drives home the point that AI is on track to become a mainstream factor in Enterprise computing. Even though the horizon of their forecasts stretches to 2020, a team of Oracle execs make up our AI optimist all-stars, providing another reason to make the house that Larry Ellison built our starting point.
Siddhartha Agarwal, Oracle's VP of Product Management & Strategy, believes that, in the future, "AI becomes the app interface" and elaborates that "...AI can predict what you need, deliver info and functionality via the right medium at the right place and time, including before you need it, and automate many tasks you do manually today."
Agrawal's colleague, Amit Zavery, who is Senior VP, Product Development, Oracle Cloud Platform & Middleware, believes that the "...central tenet of artificial intelligence--to replicate and exceed the way humans perceive and react to the world around us--is set to become the cornerstone of innovation."
Suhas Uliyar, Oracle's VP, Product Management & Mobile Strategy will see the Agrawal/Zavery predictions and raise them, proclaiming that "the majority of customer support interactions will be conducted by chatbots." Uliyar also believes that chatbots will "prove essential in reducing businesses' administrative workloads."
We're from the government, and we're here to help Dave Shuman, IoT industry leader at Cloudera (who, coincidentally, was my college classmate) sees a big future for AI in the public sector, predicting "increasing use of data scientists at the agency level to build and deploy machine learning models that will improve citizen engagement and services."
Peter Ford, Public Sector Industry Principal at Pegasystems, seems to agree, saying "AI solutions will use contextual information from existing systems either within or beyond the parent agency to support the speed and quality of citizen outcomes and interactions."
Bring me up, bring me down There are other optimists. For example, Matei Zaharia, Chief Technologist at Databricks, and one of the creators of Apache Spark, feels "Data scientists will continue to grow in number."
Splice Machine CEO Monte Zweben is an AI believer too, predicting the rise of what he calls "Online Predictive Processing" (OLPP), saying it will emerge "as a new approach to combining OLTP, OLAP, streaming, and machine learning in one platform." Zweben also believes that "AI is the new Big Data."
But that seemingly bullish statement is actually a little backhanded. As he clarifies what he means, Zweben says of AI: "companies race to do it whether they know they need it or not." Like me, Splice Machine's CEO hails from NYC, perhaps the capital of sarcasm. But he is not the only one with a cautionary message.
AI yay yay For example, over at Arcadia Data, Steve Wooledge, Vice President of Marketing, and Dale Kim, Senior Director, Products and Solutions, think AI "...deserves the same treatment Hadoop and other Big Data technologies have received lately. If the industry is trying to balance the hype around Big Data-oriented products, it has to make sure not to overhype the arrival of AI."
Adding to the chorus, Patrick McFadin, Vice President of Developer Relations at DataStax, says "AI will go deep into the 'trough of disillusionment'." And while that's true of any popular technology wave, McFadin backs up his claim with concrete observation: "The largest users like Facebook and Google...make [AI] look easy but companies without that deep experience aren't seeing the same results." To Wooledge's and Kim's point, a few years ago, that same statement could just as easily have been made in reference to Hadoop and Big Data.
The skepticism can get even more severe. Christian Beedgen, CTO at Sumo Logic, says rather categorically, that "AI will not transform the enterprise in the near future." He adds: "Previous predictions and claims about the direct impact of AI on enterprises have been overblown."Suddenly, Monte Zweben doesn't seem like the AI Scrooge any longer.
Getting down to work Jon Lee, CEO of ProsperWorks tries to counterbalance the skepticism in a constructive way, stating "In 2018 the attention will be on results, not hype. The smartest enterprises will focus on ensuring their machine learning and automation capabilities bring measurable business results..."
Continuing on the pragmatic front, Nima Negahban, CTO and cofounder at Kinetica, says "...as AI goes mainstream, it will move beyond just small scale experiments run by data scientists in an ad hoc manner to being automated and operationalized." Negahban added that next year "investments in AI life cycle management will increase and technologies that house the data and supervise the process will mature."
Ted Dunning, chief application architect at MapR, and a highly respected expert in the data world, might agree. Dunning predicts that "Machine Learning Will Go from 'In Vogue' to 'In Production'" and says "organizations will recognize that 90% of machine learning success is in the logistics (rather than the algorithm or the model)."
Even Databricks' Zaharia admits that "Generic machine learning platforms are difficult for organizations to use" but he believes that "vertical-specific solutions to common business problems will start to incorporate the newest ML techniques and transform the standard business processes."
Patrick O'Keeffe, Executive Director, Software Engineering at Quest Software, also sees a role for applied AI, and discredits "doomsday" thinking around it. Regarding the role of the database administrator (DBA) O'Keeffe states: "AI/machine learning will not eradicate the [DBA] position, but rather augment it by creating new efficiencies and freeing up time for the DBA and empowering them to assume a more cross-functional role within the organization."
Internet of prudent things Closely correlated with AI and ML are trends around the Internet of Things (IoT), and much of the prediction is around maturity and ROI.
Kinetica's Negahban predicts that "Organizations will look for / demand a return on their IoT investments" and adds that "while it is a good start for enterprises to collect and store IoT data, what is more meaningful is understanding it, analyzing it and leveraging the insights to improve efficiency." This reminds us that IoT, Big Data analytics and machine learning are rather inseparable.
The IoT enthusiasm among our predictors doesn't stop there. In an age when so any customer interactions are electronic, Ryan Lester, Director of Customer Engagement Technologies, at LogMeIn, insists that "IoT Will Save Consumer Brands." He adds that "embracing IoT at the time of customer engagement helps companies to create relationships with their customers and create an ongoing engagement that will help them better understand their customers' needs..."
Rich Rogers, SVP, IoT Product & Engineering at Hitachi Vantara (the new unit of the Japanese conglomerate combining Pentaho, Hitachi Data Systems and Hitachi Insight Group) says "2018 will be the year that IoT technologies rapidly accelerate the transformation of industrial factories into software-defined factories." He also believes IoT will enable a phenomenon where "data centers begin to transform into fully autonomous operations" (which both contrasts interestingly with what Quest's O'Keeffe had to say about DBAs and AI).
Dr. Jans Aasman, CEO of Franz, Inc, says "IoT will drive the core intelligence for Smart Cities"explaining that "IoT devices on street lights, bus stops, autonomous cars, public bikes and delivery robots will provide the opportunity to gather data about traffic, biking and walking patterns, pollution, weather, and natural disaster challenges." Pegasystems Peter Ford goes so far as to say "government will predict and act upon key life events such as births, house moves and job changes."
Cloudera's Shuman seems quite on board with such thinking. He feels that "...federal, state and local agencies will increasingly focus on driving automation and intelligence into their operations, based on all the data that is generated by connected devices across our cities."
Shuman also addresses the efficacy of IoT devices processing their own data, stating that "although processing of the data at the edge has some advantages, agencies will still need to bring a lot of their IoT data sets into a centralized data store to drive advanced analytics and machine learning."
MapR's Ted Dunning is an edge advocate, though. "We are...expecting to see full-scale data fabric extend right to the edge next to devices, and, in some cases, we will see threads of the fabric extend right into the devices themselves" says Dunning.
Deploy on the cloud that is multi Whether or not edge is "a thing," there's a virtually unanimous sense that the cloud is where an increasing number of analytics workloads will end up. And, within that, there's general consensus around the importance of "multi-cloud" requirements and capabilities. It's all based on the premise that customers, in avoidance of lock-in, will seek products and technologies that are portable between the major public cloud platforms.
The multi-cloud imperative is perhaps best summed up by DataStax's McFadin, who says "more companies will embrace the multi-cloud as competition heats up between cloud vendors and fear of lock-in becomes more prevalent." He's not alone. Sean Martin, CEO of Cambridge Semantics, says "enterprises will make fundamental changes in big data access by moving towards multi-cloud strategies encompassing several cloud vendors, enabling them to decrease costs and enhance efficiency."
Ramin Sayar, Sumo Logic's CEO, says "demand for multi-cloud, multi-platform will drive the need for multi-choice," adding "cloud users are demanding choice, which is going to drive massive growth in multi-cloud, multi-platform adoption in 2018. As a result, enterprises will need a unified cloud native analytics platform that can run across any vendor, whether it's Amazon, Microsoft or Google, including what's traditionally running on-premise."
MapR's Dunning chalks the multi-cloud phenomenon up to container technology, explaining that "rapid Kubernetes adoption forms the foundation for multi-cloud deployments." Drilling down, Dunning predicts that "over the next year, Kubernetes will be the way leading-edge companies organize and orchestrate computation across multiple clouds, both public and private." Summarizing the impact, Dunning says that on-premises computation is moving to containers too, and deems the resulting portability between clouds, and across the on-prem/cloud divide, as "real revolution."
Data protection and governance For many of our future gazers, the real trends next year may not be about data technology, so much as the protection and the governance of the data itself. Regulations, especially the European Union's General Data Protection Regulation (GDPR) are weighing heavily on many of the predictors, and not just for companies in Europe.
Dr. Werner Hopf, CEO and Archiving Principal at Dolphin Enterprise Solutions Corp., has some somber thoughts on the matter. Hopf opines: "as the May 25th deadline for GDPR regulation as defined by the European Union looms, the pressure to comply will become increasingly evident and imminent...GDPR will continue to gain visibility within the U.S. and companies will need to take swift action to sort through the tremendous amount of data they have accumulated to avoid serious fines and pressure from auditors. Failure to do so could be devastating."
There are definitely no unicorns or rainbows where thinking on GDPR is concerned. George Gerchow, VP of Security and Compliance at Sumo Logic, has even stronger rhetoric, saying "GDPR regulations will turn massive tech companies into walking targets." Gerchow adds that "it won't take long after the May 25 GDPR deadline before the...European Union cracks down on audits of big tech companies...Uber, Google, Apple and so forth." Beyond Big Tech, Gerchow predicts "financial institutions and travel companies will be next, as these types of organizations are the most globalized industries, where data flows freely across geographical borders."
One starts to get the feeling that all the grandiose predictions about IoT, AI, ML, containers, and multi-cloud implementations, all of which rely on such free flow of data, will be impeded by a reluctance to let any data move at all, for fear of regulatory non-compliance and resulting financial penalties.
Ron Agresta, Director of Product Management at SAS, thinks so, and for those very reasons. He says that "governance is a growing challenge as more data moves from on-premise to cloud locations and as governmental and industry regulations, particularly regarding use of personal data, become more pervasive."
On the bright side, Hitachi Vantara's CTO, Hu Yoshida, predicts new data governance frameworks will rise as a result of GDPR and other new regulations, as well as cryptocurrencies. And even SAS' Agresta sees a compromise between innovation and safeguards, saying organizations will need "the correct balance of 'offensive' (being agile and exploratory with data) and 'defensive' (governance and control of data) approaches to solving data-centric problems."
Kinetica's Negahban brings this sense of conscientiousness over panic home to AI, where models may be prone to error and bias, asserting that "auditing and tracking every input and every score that a framework produces will help with detecting the human-written code" that could cause "detrimental impact to human life if [an AI-based] incorrect decision is made."
Emotional rollercoaster? These predictions have run the emotional gamut. We started with optimism giving rise to euphoria, saw it tempered with contrarian views leading to downright pessimism, and finished it off with some judicious thinking about balanced approaches.
That's a productive exercise as we get ready for a new year. The Big Data analytics stack has stabilized, is getting deployed to production and is being held to truly high standards of performance and ROI. The industry will be well-served if it avoids Hadoop-style misadventures and pitfalls with IoT, AI and ML, through advance risk assessment and contingency planning.