To help you secure sharp, practical, trustworthy and innovative professionals to penetration test your business, ZDNet talked to some of the top pentesters, organizations and teams in the business -- and we boiled their advice down to ten must-know rules when it's time to bring in the pros.
Penetration testing is a crucial part of fortifying and maintaining network, IP and physical security, but as we discovered through numerous interviews, it's not a simple task to hire for. Pentesting involves giving professional pentesters permission to test and verify that new and existing systems, networks, applications and safeguards don't provide unauthorized access to malicious hackers — but pentest individuals and companies range from razor-sharp, thorough and helpful, all the way to oversold, irresponsible and negligent.
Today's attackers are devious, creative, and not held back by anything. Here are the ten most critical things you need to know in order to hire the right pentesters.
Every company and individual we spoke with emphasized that pentester communication skills are critical to assess in the hiring process, and Ronnie Flathers, associate security consultant at Neohapsis told ZDNet that communication was a pentester's "most important trait." Flathers said, "The most valuable skill a penetration tester can have, in my opinion, is the ability to easily switch from deep, technical discussions to high level concepts depending on the audience."
Andre Robitaille, Red Team Sr. Manager, Dell SecureWorks concurred. "Penetration testing results are only useful if they are acted upon, and clear communication, both written and spoken, is one of the most critical skills that a tester can have."
Sameer Dixit, Director at Trustwave explained that a top pentester must have strong oral and written communication skills. "Pentesters should be able to produce quality reports and explain their findings in detail at a technical and non-technical level."
Dell's Robitaille spelled out the fact that a pentest's value hinges on understanding the results, and making those results actionable. "A tester with a strong technical skillset and the ability to convey results in a way that non-technical management can understand, is far more useful than the best tester in the world who writes in such dense technical terms that decision makers become glassy-eyed."
Neohapsis's security consultant explained why. "Technical skills can always be taught and improved, but good communication skills are hard to learn."
During the interview, I ask the candidate to explain various vulnerabilities to me as if I’m completely non-technical. He or she has to be able to explain concepts in different terms both verbally and written.
A tester may find a critical, super technical exploit on a penetration test, but unless he or she can explain it well and outline the risk, the client will have no concept of its value or importance.
2. Beware of “secret sauce” consultants
Find out how technically well-versed your candidates are. Trustwave's Dixit said, "We look for pen testers who have intimate knowledge of enterprise development framework, networking protocols, MiTM, ARP spoofing, multi – platform system administration, password storage (LM, NTLM, shadow, etc.), database systems, scripting (ruby, python, Perl etc.) and essential security toolsets."
Ian Amit, Director of Services for IOActive stressed that it's crucial to ask about methodology. "A pen test isn’t a one-off engagement that is based on luck and magic."
Pentesting is a strange gift, but blind trust is not acceptable currency. Mr. Amit cautioned about "secret sauce consultants" saying, "If the report does not contain clear information on how to recreate the issue, and recommendations on how to mitigate the risk associated with the issue, better find another service provider. If a finding isn’t repeatable, it’s not a finding, and most likely a false positive."
IOActive's Director explained concisely that a finding, no matter how curious, "needs to be repeatable, and findings need to show systematic issues that can be addressed at the core level or through mitigating controls."
3. Get involved with the security community
What do all these hacker conferences you keep reading about in the news have to do with your organization? Everything. And it's where the people are who can prevent your company from becoming the next big name, security-breach-of-the-week smeared across national news headlines.
Security community involvement is something that needs to activated before you start looking for a pentesting hire for a variety of important reasons.
Trustwave's Sameer Dixit told ZDNet, "Employers should be active participants of local information security chapters and open source security tools development projects such as github, OWASP etc. By getting involved in the infosec community, employers will gain better insight into the top pen tester talent and can begin building relationships with potential candidates."
He added that this can also help you pull in a selection of candidates, as well as make sure you have the right fit for your jobs -- and vice-versa. "The pentester community can also become more familiar with the companies that are hiring."
4. Reputation is everything
Hiring pentesters might feel like you're hiring the most dangerous people you can find and handing them a diagram to the weaknesses in your company's armor. This is why reputation is everything. There's no better place to do gauge the reputation of the individuals and teams in pentesting you'll be getting resumes from (and seeking out) than in the various community conferences.
Every company we interviewed warned that if a your pentester comes out of nowhere, that's a red flag.
Impassioned on this point, Dell SecureWorks Red Team Sr. Manager told ZDNet, "Are they involved with the InfoSec community? Have they spoken at peer conferences like DerbyCon, DEFCON, ShmooCon, or similar conferences or BSides? Do they compete in Capture the Flag (CTF) events? Do they contribute to open source projects, write blogs, or responsibly publish vulnerabilities?"
Dell's Robitaille continued, "Obviously these show that they’re passionate and enjoy penetration testing. But more importantly, it shows that they will actively improve things or teach others, which every organization and team needs – continual improvement."
Pursuit of new skills, skillsharing, gleaning insight into problems, and knowledge seeking are the core aspect of hacker conferences and what activities are organized around.
IOActive's Amit cautioned to beware of big egos in the security scene, who might value their bragging rights in a given situation over your organization's security. "Don’t always look for the 'rock stars'. Finding a thorough and methodical penetration tester will usually provide more value than hiring the latest 'stunt hacker' that may be great on a very narrow technological aspect, but mediocre when having to deal with broader spectrum of information security, which is usually the case when a company needs pen test."
5. Technical acumen: Required
The Dell SecureWorks' Robitaille stated outright "a technical interview (or lab test if available) is critical to verifying a candidate’s expertise."
The Senior Manager of Dell's Red Team spends a lot of time hiring pentesters. He said that when you're looking at a candidate's resume, online bio or any other "about me" resources you're evaluating, ask "Do they have technical certifications, and are they entry, mid, or advanced level?" He said, "Candidates with an Offensive Security OSCE or SANS GXPN certification immediately gain my interest, because they should understand advanced concepts, at least in a classroom/lab setting."
Knowledge of the toolset is key, but Robitaille added "Résumés frequently describe experience with Metasploit, BurpSuite, and Kali, but for candidates who spend most of their time protecting assets, their experience with tools is often limited to starting scans and interpreting results. A good penetration tester, on the other hand, uses tools to speed up their testing, and can perform their job just fine without them because they know what the tools are doing behind the scenes."
Prepare to hold up your end of the deal. Carl Vincent, security consultant at Neohapsis, advises that prior to hiring pentesters organizations should "Flush out your template structure as much as possible."
He said, "The more flexible and robust your template is, the more consistent your reporting will be. Testers will produce much more results if they know that the usual findings are going to be quick to jot into a report as opposed to having to write things from scratch or morph boilerplate language every single time."
6. Well-rounded, recent experience
Pentesting must be a tough job to get into, because no company or individual we interviewed suggested anyone take on pentest hires without experience. This is not a sector where someone fresh out of hacker school is going to get a top job. Dell's Red Team Senior Manager felt it should be the opposite. When hiring a pentester, Robitaille said any legitimate candidate "should have penetration testing experience as their most recent, primary role."
Several directors we interviewed said that a pentester with "administration experience in their past" is key.
Robitaille opined that seasoned defenders -- "blue teamers" who defend against "red teamers" in attack simulations -- make for better attackers. "The best pentesters at Dell SecureWorks and other organizations have spent time on a Blue Team -- people who have managed networks or systems, or develop applications."
Dell's Robitaille explained to ZDNet, "I've met extraordinary penetration testers that don’t have the deep experience that comes from having Blue Team experience, but they are rare. Dell SecureWorks still considers candidates that only have penetration testing experience, without Blue Team experience, but often those candidates reach a limit in their technical growth."
He also explained how a pentester's holistic experience set can make the difference between hiring pentesters who contribute to a well-oiled machine -- or hiring duds that make a bigger mess of things. "When a penetration tester first gains access to a system, it’s time for post-exploit... The path ends and the wilderness begins."
Unlike a classroom or lab setting, no one is prompting the tester with ideas of what to try next.
We’ve found that the difference between a "good" and a "great" tester is that they can gain an awareness of the environment because they have the deep knowledge of how systems or networks work that usually comes from having systems or network administration work experience.
For an application tester, that deep knowledge often comes from a work history of application development.
7. Hire passionate hackers
Hiring a consultant who isn't passionate about their area of expertise, no matter what that area, or how strange the culture seems, is a recipe for disaster in any situation. For these reasons, Robitaille tells ZDNet that when he's hiring pentesters, "I start at the bottom of the résumé, because it most likely tells me what someone chooses to spend their time on, versus what they’re being told to do at their jobs. If I know that penetration testing is important to them, it changes how I view them for the rest of the process."
Mr. Robitaille expressed something we've heard echoed in every corner of infosec. He said, "The best overall testers are not the ones who only enjoy breaking things, but are people who are personally driven by the principle of improving security."
He advised, "When interviewing, ask why they are (or want to be) penetration testers. Most will say that they love a challenge or love technology and learning, but the best hires will also say that they are driven by the purpose of making things better."
8. A willingness to go off-script
Most organizations rated "creativity" as their top trait in the best pentesters. Ronnie Flathers, associate security consultant at Neohapsis, explained, "No two engagements ever go the same, and you always have to adapt to your findings as you go. Even the same vulnerability can be wildly different in two different environments, and the way you attack it will rarely be the same."
Because of this, he said "I feel that too often, pentesters become reliant on the automated standard tools, scanners and methodologies. While these are great starting points, a penetration tester is going to have to adapt to what is found and, when the tools don’t work, be able to manually test."
He shared that throwing worst-case scenario obstacles at candidates can vet out button pushers so the creatives can rise to the top. Mr. Flathers told ZDNet,
In interviews, I like to present imaginary scenarios and see how the candidate responds.
When they say that they would use Metasploit or some other tool, I tell them it doesn’t work or isn’t available. When they say they’d run a port scan, I tell them the IPS just blacklisted them. The ones who know how to manually test and get around restrictions are the most valuable.
I’m not impressed by someone who only knows how to run Nessus and Metasploit and then write a report.
Perhaps Dell's Sr. Red Team Manager Robitaille said it best. "Penetration testers must be able to go off-script when the tools break, when tools don’t work for the task at hand, or when tools are too "loud" to be covert. The more genuine attack experience a person has, the more ability they have to get creative and walk without the crutches."
9. Know that a pentest is only part of the picture
You need to tick all the boxes to make your XYZ-level overlords happy, so just hire pentesters and deliver audit reports, and security is done! Not so fast, Mr. former Target, Home Depot, or JPMorgan security decision maker.
Approaching the pentesting hiring process, and its ever-evolving landscape, as your one-stop solution is a huge mistake.
IOActive' Ian Amit explained that decision makers must think in terms of layering their security defenses. He told ZDNet, "Relying solely on a penetration test to provide a view of your security posture is like relying on a single vendor firewall to mitigate all information security risks in your company.
Always combine audits, code reviews, and pen tests in order to provide better coverage of your security posture and get a more accurate risk tolerance. Correlate findings from tests with logs from your own network to see whether you are exposed to certain attacks, and understand who you are trying to protect yourself from.
10. Don't be afraid of pentesters
The discussions you're going to have about why you're pentesting, who and what pentesters you're hiring, and what will be on the reports are uncomfortable ones -- to say the least.
These uncomfortable discussions are not just a good idea, they're necessary if your organization is going to survive. Most pentesters we talked to mentioned that a huge roadblock for them in trying to work closely to ferret out security problems was encountering resistance from the very companies they're trying to work for.
Neohapsis's Vincent didn't mince words about the problems created by exclusionary behavior. He advised, "Have an IT department that includes, not excludes, your testers."
At the end of the day, pentesters have to write reports and perform typical office functions like everyone else.
Add on top of that things like blocks of IPs required for running scans, software licenses for tools, hardware for physical and wireless assessments, and you have an IT nightmare.
Vincent strongly advises to "Treat consultants like cops," and so "if your gun locker is insufficient, they are going to go elsewhere to get tools and or equipment and hate you for making them spend their own time and money to do so."
He pointed out that a common mistake in the initial trust negotiations of the pentest hiring process is lack of clarity about the basics of remote employment, on-site work and travel. He said, "Be upfront about travel requirements. Some people don't like travel, and others don't mind living on the road. Be upfront and you'll get the right people for the job."
Above all, IOActive's Amit cautioned that a lack of understanding about threats, and who threat communities are, could create for bigger problems down the road. "Don’t protect yourself from pentesters." He added, "You are not defending yourself from pentesters, so make sure that your service providers understand your risk posture and threat model, and can more accurately simulate those adversaries."
Dealing with criminals, activists, random hackers, or state-sponsored actors is completely different, as each has it’s own techniques, motivations, resources and access to your assets.
Make sure your tests reflect that, otherwise you’ll find yourself ready for pentesters rather than real attackers, and will be surprised when real attacks come through in ways you did not plan or prepare for.
You need pentesters that are dedicated to the endgame -- your endgame, which is protecting your organization. These experts understand, and ZDNet thanks the organizations and pentesters interviewed for this list, and all those who contributed their knowledge, experience and expertise to this article.