Thoughts on QA Security Testing

In several different contexts, I've been involved in conversations concerning testing for security defects in QA. Here's my take on it.

Remember that I'm a fundamentalist. I love the American League (the Senior Circuit), but hate the DH. I believe engineers should be trained in securing, but not hacking, and I believe that coders should be trained in coding correctly, but not security. I know those sound shocking to people who are experts in security, but there's nothing different about eliminating an injection attack than there is in ensuring that the new presentation layer doesn't barf.

That being said, I don't think QA testing needs to do security testing. They do need to perform "out of bounds" testing, however. QA groups generally take the functional requirements of the application, and test to ensure that all of the functional requirements are met, but they don't test to ensure that things outside the requirements are properly handled. For example, if a field is supposed to be non-blank, alpha-numeric characters, with a maximum length of 20, they will make sure that when non-blank, alpha-numeric characters numbering fewer than or equal to 20 are entered, they do not cause the application to barf. They generally don't test to ensure that more than 20 characters cause a proper warning, or that non-alpha, non-numeric characters don't cause an application failure.

Even with a stored cross-site scripting vulnerability, a proper functional test will actually reveal that the later-rendered pages don't render properly - this is assuming you have proper functional tests. But it will also reveal when the input validation and output filtering techniques being employed aren't working - which are the real core to stopping the various types of injection attacks.

I am not advocating that later security testing be eliminated. Nor am I advocating that your developers not perform static analysis. I'm just advocating that QA groups begin testing for things that are outside the functional requirements to determine that the application fails gracefully, and that they report anything suspicious back to the development team. In order for this to work, it's critical that your functional requirements be very specific about what is allowed, and that the QA testing not only ensures that what is allowed works, but that things outside the allowed range don't work. We still need security experts to engineer the application properly and to do real security testing for logical flaws (and yes, the few semantic flaws that will fall through the previous steps) after a proper code/QA/rinse-repeat cycle.