Games development: a tester’s perspective
For the last few months I’ve been involved with a company that produces games for the Mac platform as one of their ‘testers’. It’s an unpaid role of course, but I have been able to witness the growth and development of the application throughout its various builds.
The company is Feral Interactive and the strength of the process behind their testing programme is exhaustive. Each of us were required to log any glitches we found, however small, into an online bug tracker noting our machine model name, operating system, RAM and graphics card etc.
'
Approved image use: courtesy of Feral Interactive
As the various game builds took shape we could see refinements being made and witness the interaction between other testers as they commented on various aspects of the game, some of which were even new feature requests.
Quite apart from the fact that its great to see high quality games being produced for the Mac; I have written in the past about Software Change Management in the games industry and the need for automation to control the huge binaries involved – so clearly there is a need for automation at many levels of the games development project.
Although this is pretty common sense kind of stuff, the guidelines for what make a good bug report make interesting reading. The company says that bugs need to be both ‘repeatable’ and ‘specific’ – that way the software engineers can pinpoint it and fix it.
Here’s some more from the testers’ guidelines briefing:
“Avoid cuteness if it costs clarity. Nobody will be laughing at your funny bug title at 3:00 AM when they can't remember how to find your bug.”
“One bug per report please. Completely different people typically fix, verify, and prioritise different bugs. If you mix a handful of bugs into a single report, the right people probably won't discover your bugs in a timely fashion, if at all. Certain bugs are also more important than others. It's impossible to prioritise a bug report when it contains four different issues, all of differing importance.”
“No bug is too trivial to report. Unless you're reading the source code, you can't see actual software bugs, like a dangling pointer -- you'll see their visible manifestations, such as the segfault when the application finally crashes. Severe software problems can manifest themselves in superficially trivial ways. File them anyway.”
All of the above is not to suggest that the game had a lot of bugs of course – but those elements that may have been out of line were not overlooked. We’re contractually bound from being able to mention the subject matter or name of the games we test, so all I can say is that it’s not Space Invaders.