White hat standard to out lazy testers
Penetration testers have united to design a code to lift the standard of contracted hacking.
(Glasses image by hm.matheus, CC2.0)
Penetration testers from around the world are jointly developing the Penetration Testing Execution Standard (PTES) to provide clients with a benchmark to determine the quality of tests carried out for them by contracted testers.
Penetration testers, known also as "ethical" or "white hat" hackers, are tasked with discovering information security vulnerabilities that may be exploited by an attacker.
Chris Nickerson, a Denver-based penetration tester who is spearheading the standard, estimated that 80 per cent of penetration testers do not perform even adequate tests, while charging top dollar for their services.
"Pen tests become a vortex that suck money out of people, and the reputation of the industry has gone up and down," Nickerson, who runs Lares Consulting, said.
Testers should hand a report on vulnerabilities to the client so that weaknesses may be fixed, but many of the reports are currently too simplistic or confusing.
Some of the industry's worst testers are also the biggest and most expensive, according to Nickerson. While that may generate referral business for smaller testers, those involved in the standard say it's bad for the industry's reputation.
Moreover, they say there is no discernible way for clients to determine a good tester from a bad one, creating what Bruce Schneier calls an information security "lemon market".
Smart tests
Understanding the systems to be tested is the most obvious element lacking from penetration tests, according to Nickerson, who has 15 years experience in the field.
"That means foot-printing and identifying technical systems or just identifying different routes for attack, but the way many operate, their intelligence is poor at best," he said. "Some just run Nmap and say that whatever responds is a target."
Tests should be uniform, comprehensive and process-driven, rather than focused on identifying complex esoteric vulnerabilities, he said.
Testers' reliance on the Nmap free analytic tool is what drove Nickerson to start recruiting support to develop the standard. He said an enterprise client had approached him after shelling out tens of thousands of dollars to a large penetration testing firm for what amounted to a printed Nmap report.
Chris Gatford, director of penetration testing firm HackLabs, said a successful or failed test can hinge on the skills of individuals. "Large organisations can do poor work, or they can do a great test; it often comes down to the skills of an individual and can fluctuate in heartbeat," he said.
Penetration tests ideally should be conducted with two testers so that they can bounce ideas off each other and specialise in skill sets.
Both Gatford and Drazen Drazic, managing director of Securus Global, said that the testing standard would help by introducing minimum standards, but would not create instant experts.
"At the end of the day, a check-list won't replace gut instinct," Gatford said.
The standard
The PTES this month was transcribed into an alpha-phase "mind map" that encapsulates some 1800 revisions undertaken since its development began in November last year.
Each contributor to the standard has more than 10 years experience in penetration testing, Nickerson said, and develops components according to their areas of expertise.
The first draft is hoped for release at the Blackhat Las Vegas conference in August this year. Nickerson said releasing it at the event will put it through a trial by fire and help to improve subsequent revisions.
He has called for Australian penetration testers with at least 10 years experience in the field to help develop the standard, which will help ensure it does not clash with local laws.