A Rude Awakening

Because of some recent events, it has occurred to me that my recent post about Making Security Rewarding, which included some information about metrics was not complete.

Not only will we as Web Application Security Experts soon come to a point where we must have a way to prove our value either monetarily or in image or time lost, etc., we must also learn to speak two languages.

There are lots of really great experts on Web Application Security. And if you read their sites, read the trade mags, go to the conferences and such, you're familiar with the lingo. You know all the alphabet soup. And if you're a web developer, it doesn't take long for one of us to translate flaws found into a language you can understand.

However, if you're in a consulting business, you have to sell your services to a level of people who have far more to worry about than web application security. The folks you're selling your services to don't know the alphabet soup, don't understand web application architecture, and unless you're a little lucky, might not even know what phishing is.

Same goes if you're in a corporation where application security is not the business's core competency. You're probably reporting to a manager who knows something about security in general, or about application development in general, so you can probably "justify your existence" to your manager, but how do you justify yourself to their manager?

What adds to the challenge is that the tools in the space that help with the Low Hanging Fruit (LHF) promise to be able to find 100% of the very types of vulnerabilities that we experts spend a great deal of time finding manually. And then to complicate matters worse, there are tools made for protecting security that promise to eliminate 100% of web application vulnerabilities or 100% of fraud. And as long as that vendor gets to define what is meant by "web application vulnerabilities" or "fraud", they just might.

I'm not sure I've made that last point clear. Maybe a hypothetical will make more sense:

Suppose a particular application firewall promises to eliminate all your web application security flaws. That product will be sold to a level of management that might not truly understand web applications or security. But with a promise like that, and some sexy dog-and-pony shows, many socks will be knocked off. So then you as an expert, who use a few tools, but the most important of your tools is the human brain - your own intuition, you have to make a promise that supersedes the promise of a vendor. To boot, your price structure is probably far more expensive because you're not using a resalable widget, you're selling a one-time, manual process that takes real experts to accomplish.

So how do we prove our worth? And how do we do it with a competing market that uses the same diluted (I mean, abstracted so the customer understands better what we provide - don't look behind the curtain!) terminology that we use? And how do we compare apples to apples with those automated tools without saying to our customers "you just don't understand what they sold you"? I really don't know.

The only value add that I can think of is that if your organization also has people who've been developers, and who stay up-to-speed on current development technologies, you have an additional benefit in that you can communicate real solutions to real developers, not just real findings.

What scares me about the low-hanging-fruit problem is that we have to be able to justify to whomever the additional manual resources required to find the remaining 10%. When you look at sheer numbers, I would suspect that a moderately-complex web application with no concern initially for security may have on the order of 100 semantic flaws. The tools the vendors sell (for really cheap) will get the 90% of the LHF. Our really expensive processes might find the remaining 10% (or some subset, thereof). That remaining 10% may be the most dangerous, but to somebody who isn't intimately aware of the value of that remaining 10%, it's just 10 remaining flaws - no big deal (and that's assuming you have the opportunity to show the customer what the tool missed).

I hate to say it, but as much as the tools are helping us to do our job, because we can't apply real money-saving metrics to our work, and we can't differentiate our lingo from the vendors, the vendors may squeeze us out of our very own space. For example, XSRF is a purely logical flaw. However, I'm aware of at least one application scanner that mis-categorizes a URL injection flaw as XSRF. So they can put in their sales sheet that they find the buzzword of the week, when it indeed isn't true.