Why You Should Use Multiple Scanners

It is quite common to use a single vendor for a single security need. For instance, a .NET shop might use OWASP ZAP for all of its DAST/Active scans, Puma Scan for all of its SAST/Code scans, and regularly check for updates in hopes that one of their components in lieu of SCA/Component scanning. This is hard to manage, since it is difficult to combine the results from multiple tools into a single view for managing your security risk. Using a vendor that provides multiple services can help alleviate this problem, but then you're locking yourself into a single vendor and hoping that they are providing the service they're claiming to provide, which isn't always a safe bet. In my research, I've found that the best, most comprehensive security coverage comes from running multiple scanners. Here's why:

Some scanners are just better than others

This really shouldn't come as a surprise to most of you, but in my experiments running apples-to-apples comparisons of scanners against each other, some scanners (mostly) consistently outperform others. Some scanners simply have figured out how to provide a relatively high rate of true positives while minimizing false positives in comparison to their competitors. There is some correlation between price and quality, but not as much one would think — one can usually get better results from running multiple free/inexpensive scanners than a single expensive one.

Sometimes a scanner will look for one type of issue where another doesn't

Most scanners look for most of the same types of things. For instance, the vast majority of DAST/Active scanners look for SQL Injection vulnerabilities, but only some of them will look for opportunities to inject Angular code. Here again, some scanners generally look for more types of issues than others, but most scanners look for some types of vulnerabilities that other scanners won't.

Sometimes a good scanner will just inexplicably fail at a task

It doesn't happen terribly often, but sometimes I'll attempt to scan a website with one of my favorite scanners and it fails. Miserably. Then I'll run a scan with a different scanner and get high quality results. If you're not paying attention, it's easy to look at such a "failed" scan as the original and falsely believe that you have a relatively safe site, not knowing that the scanner itself just failed. Running scans with multiple scanners can help mitigate this risk.

Scanners have different areas of emphasis

Some scanners have a different point of emphasis than others. A very common example is that scanners will advertise their industry-leading low false positive rate. Unfortunately, my experience has been that these scanners usually achieve a low false positive rate by having a high false negative rate, i.e. these scanners miss issues that should be fairly obvious. Running a scan with such a scanner by itself isn't terribly helpful if you want to make your site secure, but running such a scanner to compare results against another scanner can help you pinpoint the true positives more quickly, allowing you to focus your triage time on more questionable findings.

False negatives are a problem, for everyone

I've read many comparisons (like this one that imply that finding 100% of vulnerabilities in your website is possible if you choose the right scanner. I'd like to know how these tests are run, because in my own research using sites with known vulnerabilities, I've found that scans using the best scanners can still result in a large number (around 50% is common) of false negatives. Running multiple scanners will not eliminate false negatives – manual penetration tests will always be needed – but these false negatives will certainly be reduced.

Conclusion

If you want the best security coverage, you should run scans with multiple scanners. No single scanner can do it all.