6 Testing Methodologies

A topic which comes up regularly, is how to use tcpreplay to test products like intrusion detection/prevention devices (IDS/IPS) and deep inspection firewalls. Generally, I hear people suggest three things:

  1. Use security scanners like Nessus
  2. Use ``real attacks'' like those generated by Metasploit
  3. Use a replay tool like tcpreplay to generate attack traffic
First, let me say that security scanners like Nessus do a really crappy job of testing the effectiveness of IDS/IPS and firewalls. The simple reason is that security scanners don't try to exploit vulnerabilities because it creates problems on the network. IT managers don't like it when their servers start rebooting or routers crash, so scanners use other non-agressive techniques like banner grabbing to find potentially vulnerable systems. Simply put, these non-agressive techniques often look nothing like a real attack.

That leaves generating ``real attacks'' and replay tools.

Advantages of real attacks:

Disadvantages of real attacks:

Advantages of replay tools:

Disadvantages of replay tools;

Aaron Turner 2006-08-07