AutoML is democratizing and improving AI

There's an irony around Artificial Intelligence (AI) work: it involves a lot of manual, trial and error effort to build predictive models with the highest accuracy. With a seemingly continuous emergence of machine learning and deep learning frameworks, and updates to them, as well as changes to tooling platforms, it's no wonder that so much AI work is so ad hoc. But still, why would a technology that's all about automation involve so much bespoke effort?

The problem with all the manual work is twofold: first, it makes it almost impossible for people without training in data science to do AI work; and second, people with data science backgrounds themselves face a very inefficient workflow.

AutoML, rescue me!

That logjam is starting to clear now, though, with the emergence of automated machine learning (AutoML). A few companies, like DataRobot, specialize in it. Other AI startups, like Dataiku, H20, and RapidMiner, and established enterprise software companies like Tibco, have broad AI platforms that feature AutoML capabilities too. So do the major public cloud platforms, including Microsoft Azure, Amazon Web Services and Google Cloud Platform. There are open source AutoML frameworks as well, like Auto-sklearn, Auto-Keras and Uber's recently open-sourced Ludwig platform.

AutoML looks to be much more than a fad. In fact, a few people I've spoken with in the AI market think AutoML could be the future of AI overall. As long as you have a clean data set and you know which column within it is the "label" (a.k.a target), i.e the one whose value you'd like to predict with new data, AutoML will build a model for you with a great reduction in effort. And, sometimes, you'll get your model with no additional effort at all.

With that in mind, I thought something of a primer might be in order, and this post is my attempt at providing one. Disclaimer: although I've been a follower of AI since the 80s, and an enthusiast of data mining systems since the late 90s, I'm not a data scientist. As such, my vocabulary and explanations are not pedagogically authoritative -- in some cases they may be naive.  But what I've got here should help you understand AutoML platform capabilities, which I hope will help you judge products and frameworks on the market.

Stepwise

To begin with, consider that understanding AutoML involves becoming acquainted with the major tasks in the machine learning workflow. The rest of this post will provide an inventory of those tasks and explain which ones are commonly -- or less commonly -- addressed by AutoML solutions.

Feature work: If you have a data set and know your label/target column, that's great. But figuring out which columns in your data set are relevant to predicting the label's value, and getting them into the proper shape for machine learning models to process, is your next requirement.  After you pick the columns, you may also need to convert text-based values to numbers, impute missing values and otherwise clean them (e.g. de-duplicate them, remove nulls and blanks from them, or remove extraneous characters from them).

Admittedly, if you know your data well, a lot of this can be done manually, even without formal data science training. Regardless, data scientists will do a better job. That's why, while some AutoML platforms require you to specify your features, many will analyze your data set and suggest which columns would serve well, letting you accept those selections as-is, or modify them.

Algorithm selection: this includes determining the type of algorithm, the library/framework to use and the specific algorithm of the appropriate type, within the library. There are rules of thumb for picking the type, depending on the prediction you're trying to make and the structure of your data. From there, picking an appropriate algorithm can be done, even if imprecisely, by guess work.

But algorithm selection is where many AutoML systems shine, even to the point of automating a competition between multiple algorithms, as detailed a few paragraphs below.

Hyperparameter tuning: every algorithm has parameters that can be set -- and a range of values that are accepted for each of them -- to control the configuration of the algorithm and how it is applied to the data.  While in some cases you can go with default values, copy values from code you find online (even if that code is for a completely different problem) or simply guess, setting hyperparameter values is important work that is in no way straightforward.

That's why virtually all AutoML frameworks include automated hyperparameter tuning...even if they do it by some brute force method. Combine this capability with the algorithm selection and feature selection that many AutoML systems also do for you -- and suddenly AI becomes accessible to a much broader set of technologists.

Beauty contest

Model competition: While feature work, algorithm selection and hyperparameter tuning can be done algorithmically, generating exactly one combination of each, certain AutoML systems will pick a set of candidates for each, then build models based on various combinations of those candidates.  From there, your models are trained and tested, to determine which is the most accurate. And with some AutoML systems, the metric used to determine accuracy is configurable.

While the generated models are trained, AutoML systems typically display a "leaderboard" of the most accurate models. When all training is complete, the model at the top of the leaderboard is, by definition, the most accurate model and the one most AutoML users will select.

Could you do the competition by yourself?  Maybe, but most people -- including data scientists -- won't want to write and debug the code required to do it. Many AutoML systems will do it, though, providing important assistance to data scientists and non-data scientists as well. Such work helps assure you'll get the best model possible.

Building ensembles: creating a set of models, then packaging them up to look and behave like a single model, is a task that some AutoML systems will take on for you. Internally, what happens is data sent for scoring (executing predictions on) is run through all the models and then the prediction from each is tabulated and, according to one formula or another, a consensus prediction value is returned.

Ensembles are typically more accurate than single models but scoring takes longer, since it must be done several times and then a consensus predicted value must be computed. Much of this work can be parallelized, though the extra infrastructure required to run things in parallel isn't free.

Shipping out

Generating a model through AutoML may be impressive, but if the model is never actually used, who will care?  That very concern is why some AutoML systems will also deploy the model to production, then monitor and manage it, to maintain its accuracy and efficacy.  Let's conclude by looking at these tasks.

Model deployment: This involves creating a callable Web service (almost certainly REST-based) for scoring new data against your model, then deploying it to some hosting environment and giving you back an endpoint (i.e. a URL) where it can be called. The service will be built in such a way that the input parameters on the service correspond to feature values and the return value corresponds to the predicted label value.

Model monitoring: This involves keeping an eye on the model, by running new data against it and seeing if the accuracy is maintaining itself, or if it's decreasing. Model "drift" -- the phenomenon whereby the statistical properties of the label/target change -- can be monitored as well. Note that, as with the model competition, the metric used for accuracy can vary and is configurable.

Model retraining: Some systems will retrain models on an automated basis, either in response to accuracy dropping below some threshold, or just as a matter of policy, at some determined frequency.  This is especially relevant for models built on streaming data. AutoML systems that handle this for you are really handling the entire process, end-to-end. This is cutting edge stuff that is bringing the concept of continuous deployment to AI.

What's Next?

As you can see from the number and complexity of each of these steps, AI work itself is non-trivial, and the potential to automate many of the confusing or tedious parts of it has a lot of value. AutoML, therefore, could be the killer app that makes AI mainstream in the Enterprise.

Auto ML could also help push the envelope, allowing data scientists to move on to more complex tasks that build on top of the above steps being automated. AutoML's current capabilities are really just part of an opening move and AutoML may help AI up its game, overall.

This is an exciting time, where usability and adoption of AI could start to accelerate dramatically. So stay tuned. Some of the companies mentioned at the beginning of this post are working hard on rapid AutoML innovation.