Donkey On A Waffle
What Makes a Good Penetration Tester
Mon, 19 Mar 2007 20:41

What makes a good penetration tester? That is the question of the day. First let's define the terms:

UPDATED 3/20

According to WikiPedia a "Penetration Test" is - a method of evaluating the security of a computer system or network by simulating an attack by a malicious user, commonly known as a hacker. The process involves an active analysis of the system for any potential vulnerabilities that may result from poor or improper system configuration, known and/or unknown hardware or software flaws, or operational weaknesseses in process or technical countermeasures. This analysis is carried out from the position of a potential attacker, and can involve active exploitation of security vulnerabilities.

First let me take this definition and refine it just a bit. We are going discuss specifically "application penetration testers" (APT). What is meant by that is that the broad scope of "network penetration test" is a bit more refined and focuses more on COTS applications and custom code. Many people consider this skill set to be what a "real" penetration test is. Anyone with a basic knowledge of computers can run nessus and create a report. It takes a certain skill and mindset to test at the application level. Some would even go so far as to say that what we really are talking about is a vulnerability researcher. So what makes a good "penetration tester" as we have defined it?

Technical skills

The exact technical skills required are largely dependent upon the applications being tested. Web application testing uses a completely different tool base and has many different attack techniques than say a windows binary product penetration test. However, the basic skills are the same. Understanding how applications are built, how code works, how to properly conduct threat modeling, abuses cases, determining areas of security analysis for a given application, and finally execution and creation of proof of concept are universal. This can all be taught. People can learn these skills. It is best to be intimately familiar with the target application if at all possible. If the tester is not already an expert in the application specific components, this process of familiarity typically takes place at the start of an engagement.

Ability to learn

Since applications and custom code are always so different, an APT consultant has to have the ability to learn quickly. Gather the resources and absorb them in a very short time. This is due to the fact that penetration test work is typically timed boxed. What is meant by that is that while the hackers have the luxury of infinite time, a paid consultant is limited to whatever the client is willing to pay. This doesn't apply to vulnerability researchers conducting work on their own time, but when someone is paying you for it I can just about promise you they won't let you test forever. You have to be able to learn FAST. It has to be in the consultant's nature to want to know, to want to learn, and to want to do it as rapidly as possible. "If I don't know it, give me 24 hours.. I will!"

The competitive drive

This is key. The drive to win. The drive to find a hole in everything you touch. Many people don't have this drive. Many people will give up at the first brick wall. Someone once told me that "Every application you touch is broken. It's up to you to find out where." This attitude that everything is broken in some way and that it is only a matter of time until it's found is the drive that a good APT consultant needs. This personality trait might best be described as stubborn or obstinance. There is nothing that is going to stop you from finding that vulnerability.

Creativity

I received a little feedback from Curq of 0x90 fame today. He pointed out another trait that indeed is something that is required of a good penetration tester. Creativity. A good penetration tester must be able to create, invent, and otherwise imagine new threats and attacks. Things aren't always going to follow the same old threat path, and invariably the attacker must figure out new techniques. That is where creativity comes in to play!

Experience

Last but not least is experience. The more an APT consultant has, the better they are going to be. It's just like anything else, the more time you have practicing the stronger your game will become. Penetration testing is no different. This isn't a requirement to becoming a strong penetration tester, however it is a good reference when looking to see how strong a tester may be.

That's about it. If a person has strong technical skills in the right areas, the ability to quickly learn, and that all important competitive drive, they can become a strong penetration tester. All that is required at that point is experience, and that just comes with time.

----

Other areas of interest:
-What is the best way to teach someone to find vulnerabilities?
-What is the true value of a penetration test?
-Penetration testing is a function of time. Is there some magic amount of time that is typically enough? How do you properly scope an application penetration test?
-Is it possible to accurately quantify risks attributed to vulnerabilities discovered from a penetration test?

Comment and discuss!