commentary The formation of an Australian chapter of the Council of Registered Ethical Security Testers (CREST) is a great thing for the industry overall, but could we lose some of the brightest minds simply because they think outside of the box?
Thanks to the new Australian organisation, which was announced by the attorney-general last week, Australian penetration testers finally have a formal body through which they can be represented. However, it does raise a fairly subjective question of how good a penetration tester needs to be, or even if the quality of one can be meaningfully measured.
In particular, CREST has an assessment that testers must pass if they want to earn their rank as a certified or registered tester. The current test for other countries already participating in CREST is a two-part practical assessment and multiple choice exam.
The syllabus (PDF) shows that there are a large range of topics covered in the assessment, but it could be argued that assessments often fail to test or consider the real-world ability of a tester to think "outside the box".
For example, as detailed as the syllabus is, it doesn't test an individual's ability to identify lapses in physical security, or how, combined with an understanding of RFID (Radio Frequency Identification) or NFC (Near Field Communications) access devices and how they or other smart cards could be compromised, someone could walk a machine out of a building.
The technical enthusiasm and drive to stay on top of the game that penetration companies look for in recruits is also difficult to test for. Do you ask a tester what was in the latest issue of Phrack to see if they keep up to date with what is happening? Do you ask them to name their top 10 sources of information?
A reformed black hat could also be potentially the greatest asset to a penetration testing company due to their past experiences and ability to "think like one of the bad guys", but no exam would be able to assess whether they were likely to go rogue. A rogue hacker working under the guise of being a certified tester would be a venerable position to be in if not just for bragging rights.
There's also a certain sense of irony in providing penetration testers with certification considering that so many penetration testing companies have become wary of judging potential hires on certifications alone. In many cases, certain certifications have become devalued due to the ability of candidates to purchase them, or pass the exam by selectively studying content a day or two before.
That isn't to say that certifying testers isn't appropriate — there are many benefits to having a representative body for the industry, especially if complaints about a poorly performing member could result in their expulsion or other disciplinary action. But whatever the approach to addressing the industry's issues, it should be careful not to penalise those that do a good job.
Historically, some of the world's best minds have flunked through structured learning — Apple co-founder Stephen Wozniak was a college drop out who discovered the joy of phone phreaking with Steve Jobs, and Microsoft's Bill Gates, who found himself hunting bugs while in high school just so he could use computers, is well known for dropping out of Harvard University.
In particular, information security seems to be one of those areas where a mind that doesn't necessarily conform to strict standards is sometimes one of the best at finding the odd quirk. It would be a shame to potentially discount someone on the basis that they simply don't fit the cookie-cut build that an assessment might enforce.