Big data burn-out looms as data scientists hit by work-related stress

A mismatch between workers' personalities and the demands of the job is causing problems for data scientist, according to research.

Data scientists are over-stressed and risking burn-out because they're being forced into roles that don't suit their personalities.

A survey of 600 data scientists in the UK and Ireland found that many are "exhibiting high levels of work-related stress", with 27 percent of male data scientists saying they were "mildly stressed" and 25 percent said they were "heavily stressed". For female data scientists those figures rise to 28 and 30 percent.

The report by data analytics company SAS put much of the stress down to a mismatch between workers' personalities and the demands of business. The respondents completed a psychometric survey which was then used to define the personality of the typical data scientist and to compare that to the skills required by organisations.

The research identified ten psychometric profiles present in the data scientist community. While technical, analytical and logical skills still dominate in the profiles, other skills such as project management, creativity and good communication skills are also present.

The report said organisations must better identify and define what they need from data scientists. As it's unlikely that any individual will have all of the skills required to maximise the value of big data, managers need to identify the particular skills they need and build a team that includes them all.

Wanted: Big data unicorns to wrangle unruly data

Wanted: Big data unicorns to wrangle unruly data

Big data skills remain hard to find, and confusion about how to define what a data scientist does is just adding to the headaches.

Read More

It warned: "Failure to do so can result in individuals trying to fulfil roles to which they are not suited – which may lead to stress and burn-out."

Data science is a relatively new concept, and as such organisations are struggling to hire and develop people with the appropriate skills and experience. Indeed, workers with the right skills have been described as ' big data unicorns ' because they are in such short supply.

Three-quarters of survey respondents claimed they have fewer than ten years of experience in data science, and nearly half had only been on the job for three years. As a result, workers can find themselves in roles their personality type is not suited to.

The survey found that a high proportion of the data scientists surveyed are "regularly adopting behaviours that do not match their personality profiles" – changing their behaviours to match what they believe is needed in their job.  

"While this is common in fast growing specialist disciplines like data science, substantial amounts of behavioural adjustment are not sustainable and can lead to burn out or a failure to realise an individual's potential. This is evidenced in the high levels of work-related stress indicated in our sample," the report warned, and said that pointed to four classic origins of work related stress:

  • Communication issues with their direct manager
  • Having a poorly defined role: often due to new activities being officially or unofficially added, leading to no objective means of measuring performance
  • Responsibility without authority: leading to no direct control over factors that affect an individual's ability to perform the task at hand
  • Lack of skills: either because the job is too difficult or because a person has not yet been adequately trained.

"It's clear that more focus is needed on defining the various job responsibilities within data science so managers can recruit individuals with the appropriate personality types to fulfil the role without undue stress. The development of well-defined benchmarks should be a priority, as it will likely lead to increased job satisfaction and levels of productivity in the long-term," the report warned.

Further reading on data science

Show Comments