The ups and downs of life with Linus

Linux kernel maintainer Alan Cox has given an insight into the experience of working with Linus Torvalds, the founder of Linux who maintains its development kernel.

During a talk last weekend at the Free and Open source Software Developers European Meeting, FOSDEM, on the challenges of maintaining a stable Linux kernel, Cox revealed that although Linus is good at developing code, he does not enjoy some of the other jobs that go along with software development such as bug fixing and beta testing.

"Linus is a good developer, but is a terrible engineer," said Cox. "I'm sure he would agree with that."

Cox explained that he and Torvalds sometimes have different approaches to fixing a problem, due in part to their different responsibilities. As the maintainer of the development kernel Torvalds needs make sure the kernel code is easy to maintain, while Cox is more interested in kernel stability and is not so worried about "hacking" the code to get it to work.

"One of the hard problems to fix are design errors," said Cox. "These are a pain because they need a lot of refactoring. Linus' approach is to re-write it to a better design. But to get a stable kernel you tend to do small horrible fixes. Linus is very keen to have maintainable code, while to have a stable kernel I'm keen to have code that works."

Cox said that Torvalds does not always let people know when he has fixed a security bug in the kernel. This can be a problem as the patch will take a while to make it to production, which means that hackers can exploit the vulnerability before it is made available to individuals and enterprises running Linux.

"Linus has this bad habit of fixing security holes quietly," said Cox. "This is a bad idea as some people read all the kernel patches to find the security holes."

Linux enjoys a reputation as a particulary secure operating system, compared to rivals such as Microsoft's Windows. Last month a mailing list was set up to help Linux kernel developers share information on security flaws.

Deciding what bugs to fix in the Linux kernel is not always easy, particularly as fixing it can impact other applications. Cox said he gives top priority to bugs that are reported soon after the release candidate is made available.

"Release candidates will pick out a lot of the stupid bugs, and what are plain stupid ideas," said Cox. "Two or three days after the release candidate we will have 150 emails with same bugs."

These early issues can be easy to fix as they are often obvious bugs. "Early problems you get are normally very easy to fix," said Cox. "As soon as the release comes out bug reports say 'You've broken this'. Almost immediately you go, 'Whoops, that’s my mistake'. Ten minutes later the fix is in the development tree."

But kernel bugs that appear easy to fix can be misleading. "Sometimes you see a fix and think 'this is perfect, move my fix into the kernel tree'," said Cox. "Later you think, 'I must have been drunk. Don't apply that patch'."