Machine learning opens up new worlds for developers

Survey shows continuing expansion of data scientists roles, but who's around to fill them?

The continuing -- but slow -- embrace of AI and machine learning means more work in designing and building models and underlying systems. These types of projects will increasingly be performed by IT departments, as the growth of data scientists is tapping out. 

escalator-toronto-convention-center-july-2018-photo-by-joe-mckendrick.jpg

Photo: Joe McKendrick

That's the conclusion of a survey of 750 technology managers and professionals, released by Algorithmia, which examined growth and staffing patterns in machine learning initiatives. The survey's authors conclude companies aren't necessarily ramping up on their data science staffs, but those staff members are getting busier. 

This is opening up new opportunities for those with related skills. At least 19% report having more than 50 data scientists on staff -- up from nine percent in the survey from a year ago. Those roles are growing rapidly across all industries, the report observes. With data scientists in extremely short supply, this means pressing existing staff into these roles. "The overall lack of data science resources will result in an increasing number of developers becoming involved in creating and managing machine learning models. This blending of roles, will likely lead to another phenomenon related to this finding: more role names and job titles for the same sorts of work." 

New types of AI and machine-learning jobs being created include the following:

  • Machine learning engineer
  • ML developer
  • ML architect
  • Data engineer, machine learning operations (ML Ops)
  • AI Ops

The success of machine learning initiatives depends on where one sits, the study also shows. A majority, 58%, said their efforts are successful if they produce ROI, reduce customer churn, aid in product adoption, or promote brand fidelity. Another 58% said machine learning efforts are successful when model accuracy, precision, speed, and drift meet threshold. 

These measures of relative success vary by role in the enterprise, the survey's authors report. "The individual contributor level -- data scientist, software developer -- values technical measures of ML success more so than the business metrics." At the same time, C-level executives and VPs "generally place more value on the opposite -- measuring ML success by how it ultimately benefits the company at a strategic level."

For IT and line-of-business directors, there's a bit of both business and technical metrics in play. Mid-level managers and directors value both the business unit impact -- ROI, budgetary, strategic planning metrics -- as well as the more technical metrics surrounding model performance. The Algorithmia authors predict that the manager and director level "will prove to be the crux of ML decisions made within organizations in the coming years as they seek to demonstrate their teams' capabilities but also prove to senior management that ML is a worthwhile investment to make."

(More details on the survey results reported here.)