Analytics and AI in 2022: Innovation in the era of COVID
As we have reached the end of 2021, my inbox has become stuffed with the now customary batch of emails, from tech companies and their PR agencies, sharing management's thoughts on what next year will hold for us, in the world of data, analytics and AI. As ever, the annual exercise of compiling sage predictions about the upcoming year, from executives around the industry, was a big effort. In fact, once all the prediction emails were consolidated, a 50-page document resulted. As with any big data exercise, my goal was to aggregate the data into groupings I could organize it by, both to tame the volume of the data and because the groupings are themselves instructive.
COVID-19 Testing
This year, most of the predictions were not about particular technologies, like Hadoop, Kafka or Spark. And they weren't even about tech genre issues like the battle between data warehouses and data lakes. Instead, this year's predictions (for next year) focused on broader business or even societal issues, many of which stem from the world's collective experience of the Coronavirus/COVID-19 pandemic and the changes it has imposed.
Technology's in there, of course; for example, there was a lot of discussion of artificial intelligence (AI), low code/no code platforms and architectural approaches to analytics, like Data Fabric and Data Mesh. But issues pertaining to supply chain disruptions; labor shortages and the "great resignation;" and the interplay between customer experience management, personalization and data protection had huge presence as well.
Let's explore a grouped summary of the predictions.
Supply Chain
We all know the pandemic has had a knock-on effect of impacting global supply chains. For business, that has meant disruption and flux. Interestingly, many of our prediction participants saw this as a tech challenge. As unpredictable as supply chain issues might make things feel, many predictions were based on the premise that predictive analytics could mitigate the the difficulties, as long as the models themselves were carefully monitored for accuracy and drift.
"Many industrial companies...over the last two years...were forced to rely on AI and other digital technologies to solve urgent, real-world problems in supply chains and production" said Artem Kroupenev, VP of Strategy at Augury. Nick Elprin, CEO, Domino Data Lab, feels "the continued volatility of unpredictable business factors, from supply chains to extreme weather, will greatly accelerate the need for businesses to continuously monitor how well their models reflect the real and rapidly changing world of their customers."
Alok Ajmera, CEO of Prophix, opines "Businesses have seen their share of disruptions over the last year, and in 2022 they most likely will continue to deal with the impact of supply chain shortages, rising inflation, and increased cyber security threats...Cloud-based "if/then" scenario planning software tools [enable]...continuously reset forecasts...according to real time data insights to meet fluid market conditions."
Beware, though. Supply chain issues don't just impact material goods; they can impact the technology we tend to view as infinite and always available, too. With that in mind, Lenley Hensarling, chief strategy officer at Aerospike, said "Supply chain issues, both hardware and personnel, will continue with the cloud..as companies continue to migrate to the cloud in 2022, they'll be surprised to find hardware and personnel shortages that may force them to alter their plans."
You can't fire me, I quit
And speaking of the personnel shortages, many of our predictors had something to say about "the great resignation," 2021's apparent voluntary exodus of workers from their jobs. For example, Buno Pati, CEO of Infoworks says that we should "consider [the great resignation's] particular implications regarding scarce data engineering and data science talent, already in very high demand and commanding significant compensation." In other words, finding good data pros was bad before, and it's getting even worse. As to a solution, he predicts that "in the coming year, adoption of new automated approaches to data operations and orchestration will free this critical talent pool from the mundane and focus these valuable professionals on creating and delivering business value."
Momentive (formerly SurveyMonkey) CIO, Eric Johnson, agrees that automation can help address the shortage and work as a retention tool to keep existing employees happier: "in 2020, 31% of companies had at least one system automated. With the great resignation and demand for better quality jobs, automation will accelerate immensely to remove monotonous processes and ensure employees have high-quality work experiences with diverse and meaningful tasks." Speaking to that notion of quality, Johnson also says "The Great Resignation has shown that pay is no longer a main retention lever. Employees are instead eager to have the working environment they want and new challenges that keep them improving and learning as individuals and professionals."
There's an AI angle here too. Nick Curcuru, Vice President of Advisory Services at Privitar, says "analytics in HR has traditionally been about reporting. As organizations grapple with employee turnover amidst the 'Great Resignation,' they will increasingly look to predictive analytics to help save the day." Gleb Polyakov, co-founder and CEO of Nylas, says "Advancements in AI combined with the labor shortage and demand for rapid scale have forced companies to automate wherever possible." Polyakov doesn't predict a dystopian situation though, assuring us that "...even with continued AI-led innovation, humans will continue playing an essential role in true organizational impact, analysis, and growth."
Of fabric, and mesh
Beyond societal issues, architectural ones have been important this year too, especially those that can help companies leverage their analytics investments more successfully. Leading contenders here are the Data Mesh and Data Fabric approaches and many of this year's predictions focused on them.
To begin with, Steve Totman, Chief Strategy Officer at Privitar, believes that "Organizations are increasingly embracing data mesh and data fabric methodologies as the basis of their modern data stacks." Infoworks' Pati is right there with him and thinks that "2022 will see significant growth and interest in data fabric solutions as companies seek to leverage a common management layer to accelerate analytics migration to the cloud, ensure security and governance, quickly [and] deliver business value..." On behalf of the company, he states "we believe this technology will be broadly adopted over the next five years."
Totman and Pati are not alone. "'Data fabric' will be a popular buzzword." was one of five big predictions from Mark Van de Wiel, VP of Technology at HVR (now owned by Fivetran). And Ravi Shankar, SVP and CMO at Denodo, headlines his #1 predicted trend as "Data fabric becomes the foundation for the distributed enterprise," adding that "by enabling organizations to choose their preferred tools, these data fabrics will reduce time-to-delivery and make it a preferred data management approach in the coming year."
Haoyuan Li, founder and CEO of Alluxio, sees Data Fabric as more than popular...he views it as a solution to the new round of data silos: "with SaaS and managed services in the cloud creating data silos, improved governance and catalog with a data fabric spanning multiple services will come to the rescue in 2022." Komprise president Krishna Subramanian sees a similar silo-busting value proposition, stating that "IT and storage managers will choose data fabric architectures to unlock data from storage and enable data-centric vs storage centric management."
Sure seems there's a lot of hope riding on these architectures. I wonder what next year's experts will say about that.
No/Lo-Co
We've already mentioned automation as a partial antidote to the labor shortage, and many of our predicting personnel see low-code/no-code platforms -- in many ways, automation's partner -- as a huge force for efficiency and change in 2022.
ZDNET Recommends
Christine Spang, co-founder and CTO of Nylas, predicts, given the "Engineering Labor Shortage, Low-Code Development Takes Off," and expounds on that headline by saying "we'll see the adoption of low-code, no-code tools and applications accelerate in 2022 and beyond."
Venkat Thiruvengadam, the founder and CEO of DuploCloud, explains: "no-code/low code platforms enable non-technical business experts to create software in the domains they know best, without requiring coding knowledge. Experts who were previously locked out of a business idea because they lacked the coding expertise can now get started with minimal coding knowledge." He adds that "the DevOps skills shortage will only continue to grow, accelerating the need for no-code / low code solutions."
But Gil Hoffer, Co-Founder and CTO of Salto, addresses the flip side of that DevOps observation, believing that no- and low-code will acquire attributes of conventional software: "as no code and low code platforms and tools become more pervasive, methodologies and tools from the software development and DevOps worlds, such as automation, version control and declarative languages will be applied and added to these environments."
DevOps-hungry or not, Mendix CEO Tim Srock and his team think "low-code will move past app development" and feel it will grow to "include customer experience design and intelligent workflow automation." The Mendix gang also feels that "...companies will use low-code to automate tasks within workflows." And Ryan Welsh, Founder and CEO of Kyndi, thinks low- and no-code will move into the AI realm too, predicting that "Universal AI will be embedded in a suite of configurable business solutions that do not require coding."
But how long can low/no...go? Since the advent of CASE (computer-aided software engineering) in the 1980s, we've seen platforms that require minimal programming have huge appeal. We've also seen the pendulum swing back the other way, where code-first platforms, and the skill sets needed to drive them, migrate back into high demand.
Personalized, but not surveilled
From low-code, we move to what we might call "low-data." Dr. Jans Aasman, CEO of Franz Inc., characterized the new normal of personal data analytics: "in 2022 we will see new ways for users to regain control of their data." That's great, but it's yet another disruption, forcing companies to personalize experiences without access to as much personal data. Jennifer Krizanek, President, NA and CMO of Contentserv, describes the challenge: "2022 will witness businesses strategizing on how to personalize the customer experience without breaking GDPR laws or infringing on consumers' data privacy rights." She further opines that next year "will be the year in which businesses learn how to operate, market and personalize their offerings to consumers without tracking their every move."
But how can that be done? For openers, Denodo's Shankar says that "in 2022, organizations will leverage small data analytics to create hyper personalized experiences for their individual customers to understand customer sentiment around a specific product or service within a short time window." That makes sense, but the requirement still seems formidable.
AI, as it turns out, may be the key to doing more with less. "With machine learning, companies can optimize, automate, and personalize content and message delivery timing to increase engagement rates at the personal level" says Jason VandeBoom, CEO of ActiveCampaign. And that's not just a nice-to-have according to Mendix folks. Rather they see it as a must-have: "Hyper-personalization will become the norm: As the world becomes increasingly digital, customers will expect experiences that are tailored and can adapt to their needs and desires in the moment. To do that, applications need to take advantage of AI versus executing simple rules."
Vertical AI
AI isn't just for personalization though. In fact, the sense of the predictors' room is that vertical industries, from finance to healthcare, will benefit from AI in 2002. Ajmera at Prophix headlines one of his predictions as: "The Era of AI-Powered Corporate Finance Is Here." His analysis says "...as businesses emerge from the volatile pandemic period, expect to see CFOs finally taking the plunge into AI-powered finance technologies, kickstarting the next era of super-charged corporate finance." Franz's Aasman says "This 'Total AI' is swiftly becoming necessary to tackle enterprise scale applications of mission-critical processes like predicting equipment failure, optimizing healthcare treatment, and maximizing customer relationships."
Further along the healthcare road, Dave Wessinger, CEO and Co-Founder, PointClickCare, believes that "in 2022 we should expect to see more healthcare organizations capitalize on artificial intelligence (AI) to monitor patient behavior, overall creating a safer environment for patients and powering healthcare staff to provide more streamlined and better care." It's not just AI, either. Todd Gottula, President, Clarify Health, thinks analytics will come along for the ride: "Payers, providers, and life sciences companies will turn to on-demand analytics software that gives them instant access to a 360-degree view of the patient journey and a window into how medical care and social and behavioral determinants of health interconnect to influence outcomes."
Transformation can be fun
Virtually everything we've looked at so far pertains to digital transformation, even if only implicitly. But how well have such "DX" efforts gone this far? Not far enough, in the view of Contentserv's Krizanek, who "...believes in 2022, organizations will extend earlier digital transformation efforts but, in doing so, will be forced to rely on their ability to automate trusted data delivery."
To get there, AI comes to the rescue, again. Komprise's Subramanian believes "AI and machine learning in the cloud continue to deliver more capabilities for customers and are becoming core enablers of digital transformation." Subramanian believes it's less not only about AI though. Companies' jettisoning their proclivity to be data pack rats is another factor: "Zombie data or dead data will garner proper attention as enterprises aim to better segment, classify, organize, cleanse, manage and justify spending on storage, backup and DR. Data hoarding will come to an end as part of successful digital transformation initiatives."
SingleStore's Oliver Schabenberger thinks DX success is a data storage technology issue, and one that will be solved in a couple of years: "By 2024, data technology will have evolved to allow organizations to optimize for frictionless digital transformation rather than optimize for read/write of transactions or efficient scans of large datasets. Databases will be an active participant and orchestrator of decision support." Given the company's combination of row store and column store technology in one database, this vantage point makes sense.
Don't just innovate, operate
Overall, while they don't come right out and say it, our clairvoyants seem to see 2022 as a year of fit-and-finish, rather than one of raw innovation. We already have tons of great analytics and intelligence technology, and we have, for a few years now, understood their potential collective transformative power. What we have largely neglected, though, is achieving high rates of success in implementing these technologies and making them operational. With the economic and logistical impacts of the COVID pandemic, that's now moved from being suboptimal to being inexcusable.
Now we need to take stock of what we have, slow down on adding to it and ramp up on the discipline necessary to exploit it to its full potential. Our prognosticators think that will happen in 2022. My guess is it might take longer, but it's a good omen that people have apparently been pondering it so fully in the latter part of 2021.