Looking to hire a '10x developer'? You can try, but it probably won't boost productivity
A new study sets out to debunk the idea that an organization can boost productivity by hiring programmers who are so much more productive than others that they can single-handedly determine the fate of a business.
Developer
The idea of highly productive programmers – sometime referred to as '10x programmers' – is a contentious and long-running debate, stemming from a study in 1968 that found the best developers are about 10 times more productive than the worst.
Bill Nichols, a senior member of the technical staff at Carnegie Mellon University with the Software Engineering Institute, is the latest to tackle the question in a new paper in IEEE Software, The End to the Myth of Individual Programmer Productivity.
SEE: How to build a successful developer career (free PDF)
As Nichols notes in a blog, various studies since the 1968 one have estimated that top-performing developers are between four and 28 times more productive than average performers.
But Nichols says his study found evidence to contradict the idea that some programmers are inherently far more skilled or productive than others. Performance differences are partly attributable to the skill of an individual, he writes, but each person's productivity also varies every day, depending on the task and other factors.
"First, I found that most of the differences resulted from a few, very low performances, rather than exceptional high performance. Second, there are very few programmers at the extremes. Third, the same programmers were seldom best or worst," he explains.
He argues that these findings should change the way a software project manager approaches recruitment. For example, they shouldn't necessarily just focus on getting the top programmers to boost organizational productivity, but find "capable" programmers and develop that talent.
The study involved 494 students with an average of 3.7 years' industry experience. The students used the C programming language and were tasked with programming solutions through a set of 10 assignments.
To assess variation in each student across assignments, the study ranked all 494 students based on how quickly they finished each assignment. Nichols found that about half the overall variation in performance was within individuals on different tasks rather than between programmers.
"This result prompts us to caution managers not to rank their programmers on individual productivity because the measurements are mostly noise. Instead, it is far more useful to explore the sources of variance in programmer performance within each task," he explains.
"Our study suggests that hiring 'the best programmers' will not be as effective or simple as we might think," he notes.
SEE: Programming language popularity: Apple's Objective-C tumbles down the rankings
Which is not to say really productive developers don't exist, but that an organization will probably find it very difficult to identify and then hire them, and doing so may not lead to massive productivity gains.
"While some programmers are better or faster than others, the scale and usefulness of this difference has been greatly exaggerated," he concludes.
Other ways of raising productivity would be, for example, to keep tasks small, plan for delays and uncertainty, give developers a quiet place to work, and develop their talent through training.