On one end of the application security and IT audit spectrum we have people that overlook the obvious and critical stuff. But just as dangerously, on the other end of the spectrum we have people who want us to find every single flaw on every single page of every single website or application. This can be a tricky situation to deal with when you have management – or auditors – who expect perfection and assume that every security flaw can somehow be uncovered. These are the people who demand a clean security assessment report every time. If it were only that easy…

I once worked on a web application security assessment project where we found a dozen or so security flaws – about a third of which were considered critical. Over a year later, the client contacted us back and told us that an internal auditor of theirs came in and found another, even more critical, flaw that apparently we overlooked. They wanted to know why we didn’t find it.

After getting some preliminary information, I told my colleagues that we need to make sure that our client’s expectations were properly set by telling them there’s no way to guarantee our results in such assessments. We can tell them we’ll do the very best we can using proven tools, techniques, and some good old-fashioned common sense. However, there’s just no way to claim we could ever find every single web flaw. There are just too many variables and there’s not enough time or money to make it worthwhile in 99.9% of the situations.

Another issue that’s not talked about much is the fact that different tools find different things. This underscores why manual analysis is so important. Yet still, you’re not going to find everything. Doctors cannot (and will not) say they’ll find everything wrong with you…ditto with home inspectors and other professionals. It just isn’t going to happen.

It ended up that our client’s internal auditor found their new flaw by running their scans over the local area network, not from the outside as was originally assumed. So thanks to their firewall the vulnerability wasn’t accessible to the outside world. We were off the hook.

As you go along in your web application security testing program, it’s important to point out to the decision-makers the fact that even if you were able to find every single flaw, there wouldn’t be enough time or money to fix it all. Striving for web security perfection is futile and will set everyone up for failure long term. Instead, find the low-hanging fruit and fix it along with the underlying operational or political issues that let it blossom in the first place. That’s a no-brainer that many ignore.

Once you have the basics under control, then proceed to find the more complex flaws that are creating additional business risks. Odds are great that anything beyond that is time, money, and effort poorly invested.