'Learning' is still the operative word in machine learning initiatives

The hype machine for AI and machine learning has been going full throttle, and one can be forgiven for thinking that every organization from the mega-techs to the corner store is turning over processes or decisions to AI. If you're still stuck trying to figure out how AI and machine learning can fit into your operations, don't worry -- so is everyone else, actually. Companies may be increasing their investments in machine learning and machine learning development, but, for the most part, are still in the early learning stages. 

That's the major takeaway from a survey of 750 technology managers and professionals released by Algorithmia, which specializes in such things. Survey respondents represent companies that are actively engaged in building machine learning lifecycles. While companies are increasing their machine learning investments, challenges ranging from model deployment to scaling and testing persist. (The impact on job roles explored in this related post.)

The report's authors point to challenges in moving machine learning forward, including scale, version control and model reproducibility, as well as getting senior executive buy-in. In addition, time is often not on the side of ML projects. "At companies of all sizes, data scientists spend at least 25% of their time deploying models, which indicates that a quarter of data science capability is lost to infrastructural challenges. Data science teams need to be able to deploy their work as quickly as possible to prevent their insights from being overcome by events; models and data change quickly as do market opportunities. As such, an insight that comes 10 days too late is overcome by events, and no longer useful." 

The report points to some of the various components of AI and machine learning in play, such as computer vision and new iterations of machine learning frameworks, including PyTorch 1.0, an open source machine learning framework, and TensorFlow 2.0, the open source framework originated from Google. The report's authors even mention some machine learning involving quantum computers taking place, but that's getting too far ahead of things. Hardware for AI and ML applications is also progressing, the report adds.

While the tools are falling into place, machine learning development remains in the early stages in most enterprises, the report's authors reiterate. "The model deployment lifecycle needs to continue to be more efficient and seamless for ML teams."  Only nine percent of companies report having "sophisticated models in production" for at least five years, 5+ years, up from five percent in last year's survey. Currently, another 15% indicate they are "mid-stage adopters," with models in production for between two to four years. Many of these leaders are software and IT firms, the survey shows.     

Companies with established machine learning deployment lifecycles are benefiting from measurable results, including cost reductions, fraud detection, and customer satisfaction."  Cost savings are seen as the main benefit of machine learning, which likely is based on less programming time involved in adjusting algorithms, as well as reduced payroll spent for manual tasks. Emerging in second place are the benefits the marketing and sales departments are pushing for, including generating customer insights and intelligence, as well as improving customer experience. "Undoubtedly, there are countless ways companies can apply machine learning to a particular business problem," the report's authors indicate. "For example, ML can be used to run prediction modeling to make assessments about customer churn, or to apply natural language processing to millions of tweets to analyze positive sentiment."

There are two metrics that are being applied to measure success in AI and machine learning initiatives, the study's authors find:  traditional business metrics, such as guaranteed ROI, and a more technical evaluation of machine learning model performance. A majority, 58%, said their efforts are successful if they produce ROI, reduce customer churn, aid in product adoption, or promote brand fidelity. Another 58% said machine learning efforts are successful when model accuracy, precision, speed, and drift meet threshold.