I deal with infrastructure and application security testing on a regular basis. On the infrastructure/network side, the consulting and testing market is much more mature, definition of pentest and vulnerability assessment are industry accepted. It is easy to communicate with other folks about the work involved. On the application side, things are not as well defined. It will be at least a couple more years before the definition or an “application pentest” is accepted.
What is vulnerability assessment?
According to Wikipedia, “A vulnerability Assessment is the process of identifying, quantifying, and prioritizing (or ranking) the vulnerabilities in a system.” In short, it involves anything to determine if there is a weakness or vulnerability in the system subjected to the assessment, then report on it. For application testing, you would throw some test input at the application or try a number of test cases and see if it is vulnerable to any of the vulnerabilities you are testing for.
In general real world terms, the tester for a VA (vulnerability assessment) is expected to perform the reconnaissance phase which allows the tester to understand the application well enough and determine if there are any short cuts to compromising the system. Also, gathering enough data about the application (such as platform it is running on or what other virtual hosts are running) to allow later testing phases.
Then, the tester is expected to map out the application and understand the application flow and relationship between objects in the application. Some of the vulnerabilities such as business logic flaws may also be revealed at this phase. Following mapping is the discovery of vulnerabilities, for input related flaws it might involve automated tools and manual validation test. There are also various other test cases that need to be manually test, especially on the session related and access controls related flaws that are not easily automated.
In general, testers follow a common testing framework such as the OWASP Testing Guide, to ensure sufficient coverage of vulnerabilities during the process. After the discovery, the vulnerabilities are evaluated and usually manually verified again. Then a risk rating is given to each vulnerability to be included in a report.
Running a vulnerability scanner against a web application is a form of vulnerability assessment. It is also a form of assessment that is not very complete or thorough, in general, an automated scanner covers about 50-70% of the vulnerabilities in a given application.
What is penetration testing?
Penetration testing or “pentesting” includes all of the process in vulnerability assessment plus an important extra step, which is to exploit the vulnerabilities found in the discovery phase. You may ask, “Just a one step difference?” Pretty much, but this one step could separate the boys from the men. I often tell the students in my pentest class that it is common for a pentester to spend 20% of his/her time locating a single vulnerability and then 80% of the time is spent exploiting that vulnerability. The process of exploitation usually involves a lot of trial and error and may not work the first time. Depending on the type of vulnerability being exploited, some other system general knowledge maybe required to aid the exploitation process.
The better pentesters don’t usually stop at exploiting one single vulnerability. For example, a single CSRF vulnerability can be somewhat limited, bundle that with a XSS vulnerability and you have a much bigger problem at hand. In a lot of cases, an expert pentester can leverage two or three low to medium risk vulnerabilities and turn the result into a critical exposure.
The added benefit of a pentest is able to see the vulnerabilities being put into active exploitation and show the actual maximum effect.Due to the nature of pentesting, the exploitation does not really have any established framework. The exploitation is highly dependent on the skillset of the invidual/team performing the test.
An example to show the difference
Let’s use an example to illustrate the difference. Let’s say the tester is testing for SQL injection and a single quote (’) is put into all input field. In a particular field, when a quote is put to the field, a SQL error is generated in the resulting page like this, “You have an error in your SQL syntax near ‘\’0′ at line 1″ This is tell-tale sign of error SQL injection. A vulnerability assessment might just do a bit further validation such as trying to dump current user name to validate the vulnerability and then goes into reporting.
A pentest on the other hand would likely be taking a lot more time on this error alone. The pentester would figure out how to tag on extra logic or command structure into the current SQL statement so that the tester can control the SQL database. If possible, the tester will enumerate the database structure and possibly dump the whole database content. If the permission is not set properly, the pentester may also be able to jump into OS command context and start executing commands in the OS. Obviously, all these attacks requires patience and takes a lot of time to succeed.
What’s more popular?
If there is such as difference and pentesting is so much more in demonstration, why don’t we just do pentesting then? Well, there is always a costing difference making pentesting significantly more expensive than a vulnerability assessment. In fact the market is currently leaning towards pentesting; those who are concerned about web app sec are willing to spend the money to get what they think is the best. (cost more, it must be better) In the next few years, as the general public are more educated about security testing for web applications, I am sure the market will adopt both services – vulnerability assessment and penetration testing. Until then, I have to be very careful about listing requirements and looking at quotation for security testing consulting work.
