Some Things are Easier to Fix than Fight

I posted some time back about how some things are easier to fix than find. That's particularly true of application testing. It's difficult to know if you've reached every case, every role, every permission set, every set of circumstances to cover the entire application. Sure, you can know how many form elements there were and if you tested every one, or if you tested every submission or query circumstance, but that's only of what you were able to crawl.

All that being said, when doing static analysis, sometimes it's easier to fix problems than to fight them. Developers understandably are very defensive of their work. They take pride in what they do, and while most accept constructive comments from one of their kind, it's certainly very difficult for them to take arguments from a "security specialist" about how they wrote their code wrong. I suppose professional athletes are good examples - when criticized for making mistakes during games by the media, athletes (and coaches) frequently respond with a type of remark that should the roles be reversed, the media folks wouldn't make any better decision.

So when you're armed with only a static analysis, which really only has theoretical findings, not validated findings, it's not unusual to get some push back. "That couldn't really happen", "we don't allow that type of interaction", "you can't prove that", or some version of "you don't understand our code". I'm sure these all follow The Five Stages of Application Security Grief, but when you witness them all at once, the distinct stages are sometimes hard to identify.

So what's a "security professional" with "no understanding of our code" to do? Well, you have a few options. You could just ignore your customer and hope the findings really are bogus. Meanwhile, they'll continue making the same bad mistakes in other applications (and probably write books about it). Or you could continue in a futile screaming match about who's right. Maybe the one who can yell the longest will win. Or maybe, if you really do understand their code, you could provide simple solutions to their problems. It's sometimes even faster to write a solution to a theoretical problem than it is to argue the point over whether the problem is theoretical or actual.

Now, I understand that nothing goes directly from concept to production, but it might just be a step to bridge that gap between "just security professionals" and application experts. Meaning, you might just show that you're "one of them" after all.

Groovy Monkey and Syntax Checking

Groovy Monkey is an Eclipse plugin which allows you to quickly script plugins for Eclipse. While I'm not that interested in making a whole bunch of plugins for Eclipse, I can see how this can be useful. I've found it, so far, to be very handy for doing quick syntax checks on code. The better static analysis tools out there do data flow analysis to reduce the potential of false positives (er - non-exploitables), but I like to be a lot more strict. There are some things that almost always end up being dangerous, but those constructs won't end up in a static analysis. If you're trying to convince people to always use <c:out />, then <%= shouldn't be used. So just messing with the example script gives me the ability to add that little check.

Next I'll add in checks for potentially bad data access mechanisms (createStatement, executeQuery(String), etc.) just for flagging for the developer to keep an eye on. The beauty of this is that the results end up in My Tasks instead of some separate perspective.

And yes, for anything but the <%=, I could just write a semantic rule, but this will end up in my task list, and is generally far quicker.

Some Things are Easier to Fix than Find

Okay, I've not gone off my rocker. It's almost true.

When performing an application test, you might not find every single instance of a particular vulnerability. Due to time, tool, or other resource constraints, some things simply slip through the cracks. A common response to this is to enumerate all of them that were found with a great big note to the developers that this is a pervasive issue, and that a global policy needs to be adopted to fix them all. And of course, there's always the pushback - "we'll fix exactly the instances you find."

This is where I think "old school" static analysis far outshines the new fangled static analysis engines. With a really good developer, grep, and a hammer, fixing semantic flaws really comes down to a few short steps:

1. Identify the common idioms used that result in "bad things". These will differ from environment to environment, which is why you have the sharp developer. Some examples:
1. <%= %> tags
2. Connection.createStatement()
3. .exec()
2. Grep the entire source tree for those idioms.
3. Replace. The examples above become:
1. tags (in Java. And I realize there are other things you need to do to make that work)
2. Connection.prepareStatement() and PreparedStatement.set...()
3. Get rid of it

Again, that's a really, really broad strokes example. But although a lot of shops use bad API's, they use them in a way that's consistent. So the replacement is consistent. And it's quicker to grep for those cases than it is for an assessment team to perform a complete deep dive on every single form element and known values that go into the application. It's next to impossible to cover every edge case in a running application. Tthere are tools that can help you do that, but they're not free, and you still have to perform all the assessment work to determine that it was covered. Why not just understand the set of API's generally used, fix those, then do a good code review on the remainder, possible uncovering some additional API's that can be grep'ped.

Full-Disclosure Paper Edition?

While the cats over at sla.ckers (and others) do a great job of finding real vulnerabilities in real applications, we don't actually spend much time calling out book authors for the same.

A couple of guys had a presentation at a recent conference (I won't mention names because they certainly weren't the first to call out a book for telling people to do stuff in a silly way) where they followed design patterns from books on web application coding and ended up with a really horrible (in terms of security) application. They were gutsy enough to name the books they used for the design patterns. But this is really rare.

It seems that while we want for developers to be trained as real engineers who apply real engineering and computer science principals to problems, many developers writing real-world, high-visibility applications were mostly trained from bad tutorials on the internet and books at the local bookstore.

I churned on the idea of setting up a full-disclosure paper edition website, modeled after the other full-disclosure sites, but then there are issues with copyright violation and such - whereas with reporting web vulnerabilities is mostly a question of ethics, I think full-disclosure in a paper edition would be a question of copyright issue and legality. While I'm all for publishers and authors being accountable for the material they publish (hey, bloggers too - call me out when I'm wrong), I can't in good conscience stand up a site that I don't know the legality of the material that would end up on it.

That being said, do we take programming examples with a grain of salt? Do we look at them with the same scrutiny as we do the sites themselves? Where a single vulnerability on a single website is genuine, it's also one person's mistake in the end result. But flaws in books are by people who claim to be experts in the field of development, published by publishers who vouch for the authenticity of that expert opinion, fool users into believing that the book indeed tells the right way to do something.

Is this all a result of the internet, where niche experts are considered authoritative on more subjects than they ought? Where somebody writes a few blog posts on a subject and are "discovered" by a publisher who needs to get some print out in that niche market? If that's the case, how do we convince developers to use a greater level of discernment when they do that google to figure out how to use a new API?

I've Said it Before, but...

Now, ordinarily, I hate screen-scraping. If there's any other way to get the raw data, I go there first. I go through whatever (ethical) channels necessary to get direct access to the source data, whether it be in a relational database, LDAP, XML, straight text, spreadsheets, or made up out of nowhere. I can't stand screen-scraping because screen-scraping is normally very sensitive to change. Screen-scraping HTML is not generally as bad as telnet or green-screen, but it's still bad enough that I try to avoid doing it - particularly when the HTML is malformed.

But today was another occasion where it was necessary simply because I couldn't get access to the systems I needed in the timeframe I needed. What would have made this one more difficult is that the source HTML had very few line breaks to do text parsing, and the HTML was also not properly XML formatted, and to boot, was poorly-enough HTML formatted that an event-based HTML parser simply wasn't going to work out.

Fortunately, I've done a little screen scraping with Groovy before, and this task wasn't significantly more difficult than other tasks I've done before. And again, NekoHTML came to the rescue. NekoHTML takes poorly-formatted HTML code (in this case, really poorly formatted) and balances unbalanced tags (had plenty), closes unclosed tags (had many), quotes unquoted attribute values (lots of those today), and gives sensible default values to un-valued attributes (and a bunch of those, too). What results is actually well-formed (not necessarily validating, but well-formed) XML, which you can parse with any ordinary XML parse.

In this case, I used XmlParser, which allows me to do very nice GPath queries. GPath works similarly to XPath, but allows you to find really complicated paths. For example, in English, "find me the text in all the <strong< tags that are under <a> tags such that their 'href' attribute matches this regular expression." In an event-based parser, that would take a lot of work, in DOM it would be easier, but still a lot of code, and the XPath would just be nasty. In GPath, it looks like this:

texts = page.depthFirst().A.grep { it.'@href' =~ /^.*\.action\?foo=(.*)$/ }.collect { it.value.STRONG.value }

Which is much fewer lines of code.

Now, what does this have to do with application security? For those who do black-box testing, there are times that your toolkit doesn't quite have enough in it. Your proxy is powerful, but just won't get you all the values you need. If there are special considerations you need to take in order to try brute-force authentication, or if you've found a good SQL injection attack, but the way the data comes back is finicky, scripting is often appropriate. So if you're looking for another swiss-army knife, some (understandably) are still Perl enthusiasts, (understandably) happy with Python, (understandably) infatuated with Ruby, but so far, Groovy has really been doing good work for me.

That being said, the GPath statements aren't specific to HTML - GPath works with XML, which is why you might need NekoHTML. And NekoHTML isn't specific to Groovy - it's a java library, so you can use it with your other java code and use whatever XML handling you prefer.

More on Output Filtering

Another one of the Justice League guys has posted about the value of output filtering. Now, the people at Cigital are uber smart - not because they're blogging about output filtering, but because they actually look at code from a very academic perspective. If you're not reading Justice League, you should be.

Now, I'm not sure I've ever said that you shouldn't do input validation. I hope I never have, and if I have, I apologize for leading you astray. However, the phrase I've been using in reporting for some time is "business-rule input validation, presentation-specific output filtering". A favorite line of a favorite movie of mine (PCU) is when Gutter is going to see George Clinton, and he's wearing a Funkadelic shirt:

Droz: What's this? You're wearing the shirt of the band you're going to see? Don't be that guy.

While I understand the desire to prepare data early (when it comes in), you've already manipulated it to be improper in some other context. For example, if you HTML encode data when it comes in, a month from now when you're using it in a PDF, your customers will complain about all these &lt; in it.

And the very best thing about output filtering is that it can be a habit. Scott hit the nail on the head in his post - the way that you properly get input squared away varies all the time, but output filtering for each presentation layer is always the same. In HTML/XML, using .encodeAsHTML() or <c:out /> or Server.HTMLEncode becomes a habit, just as much as the style guides you expect your developers to adhere to. And it's something that can be checked with a couple of regex's, instead of a really expensive tool (not that those really expensive tools aren't good for other things).

Thoughts on QA Security Testing

In several different contexts, I've been involved in conversations concerning testing for security defects in QA. Here's my take on it.

Remember that I'm a fundamentalist. I love the American League (the Senior Circuit), but hate the DH. I believe engineers should be trained in securing, but not hacking, and I believe that coders should be trained in coding correctly, but not security. I know those sound shocking to people who are experts in security, but there's nothing different about eliminating an injection attack than there is in ensuring that the new presentation layer doesn't barf.

That being said, I don't think QA testing needs to do security testing. They do need to perform "out of bounds" testing, however. QA groups generally take the functional requirements of the application, and test to ensure that all of the functional requirements are met, but they don't test to ensure that things outside the requirements are properly handled. For example, if a field is supposed to be non-blank, alpha-numeric characters, with a maximum length of 20, they will make sure that when non-blank, alpha-numeric characters numbering fewer than or equal to 20 are entered, they do not cause the application to barf. They generally don't test to ensure that more than 20 characters cause a proper warning, or that non-alpha, non-numeric characters don't cause an application failure.

Even with a stored cross-site scripting vulnerability, a proper functional test will actually reveal that the later-rendered pages don't render properly - this is assuming you have proper functional tests. But it will also reveal when the input validation and output filtering techniques being employed aren't working - which are the real core to stopping the various types of injection attacks.

I am not advocating that later security testing be eliminated. Nor am I advocating that your developers not perform static analysis. I'm just advocating that QA groups begin testing for things that are outside the functional requirements to determine that the application fails gracefully, and that they report anything suspicious back to the development team. In order for this to work, it's critical that your functional requirements be very specific about what is allowed, and that the QA testing not only ensures that what is allowed works, but that things outside the allowed range don't work. We still need security experts to engineer the application properly and to do real security testing for logical flaws (and yes, the few semantic flaws that will fall through the previous steps) after a proper code/QA/rinse-repeat cycle.

Radeox Wiki Rendering

My apologies if the link doesn't work.

In previous posts, I made a recommendation that you use a wiki markup library as an alternate method to allow your users to enter formatted text instead of HTML, in order to reduce the possibility of cross-site scripting. And I mentioned that I did a cursory look for one and couldn't find one.

So I did another search tonight, and Radeox, has been made a separate library from SnipSnap. And to further add to my excitement, there's already a plugin for it for Grails. The plugin comes with example domain classes, controllers, and views for an uber-simple wiki. But you don't have to use it for a Wiki - you could certainly use it for just more simple rich-text editing.

Now, it appears that SnipSnap development has stalled (stopped) and the one viable fork has very little available other than some initial mods in subversion, but nothing on Sourceforge, and no real momentum (yet). If that's the case, (the SnipSnap folks say this shouldn't be the case), that might explain why Radeox's site is not responding for now. I hope this doesn't go away.

I would like to be able to snap a WYSIWYG editor into Radeox, but the one open source WYSIWYG editor that's not wired into an existing project is FCKEditor, but it doesn't appear you can change what the markup is - it's only HTML, which would require you to expect XHTML in order to do really good input validation to ensure that no scriptable attributes are included (not only event handlers, but styles that allow javascript, etc.).

So take a look at the Radeox plugin for Grails, and hopefully somebody with some time will begin to pick up Radeox and or SnipSnap again.

Book Review - Secure Programming with Static Analysis

I mentioned a couple of weeks ago Brian Chess and Jacob West's (of Fortify Software) new book Secure Programming with Static Analysis (Addison-Wesley Software Security Series). When I got home the day I mentioned it, I was giddy with excitement because I had just received my copy. I'm kinda' new to book reviews, so please bear with me.

The best and worst thing about this book is its scope. It's great because it covers static analysis from soup to nuts, including why an environment should adopt a static analysis program, how static analysis works, the types of flaws that static analysis finds, how to correct flaws found by static analysis, and how to implement static analysis in your environment. However, that's also the major shortcoming with it. Because it's so broad, it gets very technical in some points, and if you hand the book to an executive type hoping they read chapters 1, 2 and 3, you'd better hope they don't thumb to chapter 4 by accident or else they'll quickly dismiss the book. Note to publisher - make chapters 1, 2, and 3 a removable pamphlet.

Jacob and Brian were fortunate enough to have the foreward written by Gary McGraw, whom I have a great deal of respect for. However, his very first illustration is correct only up to a point:

By contrast, on the first day of software engineering class, budding developers are taught that they can build anything that they can dream of. They usually start with "hello world."

The point of the paragraph (that engineering classes start with keeping people alive in mind, while development begins with features in mind) is spot-on and well taken. However, there's a very large segment of people writing code today who did not study software engineering, but started as hobbyists who learned HTML, then decided to pick up a PHP book - chock-full of really bad examples.

Part One of the book is an overview of static analysis - why you should do it, the different types of static analysis, and some really in-depth coverage of how static analyzers work - obviously they're experts on the matter, and the coverage is very good, but will be mind-numbing to those who don't actually study software. They make very compelling arguments for not only including static analysis as part of your development lifecycle, but also including it early in the lifecycle, and making sure that developers are performing the analysis as well.

Part Two covers some of the problems commonly uncovered by static analyzers. They have many examples of bad code, but thankfully, almost as many examples of specifically how to correct it (after all, I'm about solutions, not problems). They discuss input validation, buffer/integer overflow, and errors and exceptions. Brian and Jacob have some very good examples from open source software, and the fixes that were put into place, as well as some really good recommendations for additional libraries to use that will help alleviate some of the common mistakes that lead to buffer overflows.

Part Three covers environments or types of applications that are currently being written and some of the specific types of flaws that come up in those applications. They cover web applications, XML/web services, privacy, and privileged programs. The book does a really good job of covering the Struts Validator, but I see this section as needing a bit of work - while Struts is still probably the most widely used MVC architecture, the Commons Validator can be used in webapps that are not Struts, but there's no mention that you can actually use the validator outside of a Struts application, although using it within Struts makes it much simpler.

Part Four is Static Analysis in action. There are exercises to walk you through installing Fortify SCA (there's a demo edition included), performing the analysis scan, analyzing the results, and even writing your own rules. I've not walked through the examples, but anybody wishing to see what is involved will actually want to step through these processes.

I know that Brian and Jacob spent a great deal of time on the book. They bring a wealth of knowledge to something that just doesn't get covered much in books, yet it's a critical part of any mature security or development program. Static analysis does not cover every vulnerability possible, but the ones it does catch it catches earlier, and with greater depth, than other methods.

I certainly don't agree with Brian and Jacob on every one of their philosophies. I've had the pleasure of debating with them on some of those philosophies, and I'm convinced I'm still right. Fortunately, those issues don't come through in the book, and as a general reference for executives who need to be convinced of the need of static analysis in the lifecycle, to security practitioners who need to know how to write rules for their system (of course, there's a bent to Fortify SCA), to developers down in the trenches who need to understand what their tool is telling them, this book excels.

Additionally, of the technical books that I've read in first edition, this one is probably one of the best. I've seen very few grammatical flaws or even awkward sentences, and only a handful of code flaws. The writing, you can thank the editors for. The code example excellence, you can thank Brian and Jacob for - and yes, they still write code.

httpOnly filter in Groovy

I was able to make the httpOnly filter work in Groovy - the idea was really appealing to me. However, it turns out that it doesn't syntactically save just a whole lot of effort. While groovy has duck-typing, it doesn't get pushed down to Java, so you still have to type everything within the Filter. Furthermore, because it has to be recognized when the app loads, not dynamically, you still have the same compile/reload cycle as you do with a filter written in Java. The only syntactical advantage (that I've seen so far) is being able to short-circuit some of the tests because of "the Groovy truth" and when rendering, you get to use GStrings instead of .append over and over.

So there you have it - it's not that much cleaner to implement the filter in Groovy. I'll have to check the Grails API to see if they pass you a true HTTPServletResponse, or if there's any possibility of wrapping the response that gets injected into controllers to make the cookie re-writing simpler.

Servlet Filter for httpOnly

Well, zot! It turns out that you can still get at the cookies by XHR's getAllResponseHeaders().

But since it's still not really that harmful to add it, I've thrown together a servlet filter for adding the attribute when addCookie() is called. Except for the httpOnly attribute, this is no safer than the existing addCookie - meaning if you had header injection flaws before, they're still there. Also, I didn't read RFC 2109 really carefully, so this may break version 1 cookies (however, RFC 2109 cookies are still considered experimental, and you wouldn't be using them by accident).

Now, your container probably has control over the session cookies, and those get added after all the filters run, so you'll still need to consult your app server's documentation to see how to get it to set httpOnly on the session cookie (the one cookie that needs it most).

Furthermore, this does not check for cookies being added by addHeader() or setHeader(). Whether you want to add similar checking to those or not is purely up to you.

public class HttpOnlyResponseWrapper extends HttpServletResponseWrapper {
 private static SimpleDateFormat cookieFormat = new SimpleDateFormat("EEE, d MMM yyyy HH:mm:ss zzz");

 public HttpOnlyResponseWrapper(HttpServletResponse res) {
   super(res);
 }

 public void addCookie(Cookie cookie) {
   System.out.println("Adding cookie");
   StringBuffer header = new StringBuffer();
   if ((cookie.getName() != null) && (!cookie.getName().equals(""))) {
     header.append(cookie.getName());
   }
   if (cookie.getValue() != null) {
     // Empty values allowed for deleting cookie
     header.append("=" + cookie.getValue());
   }
  
   if (cookie.getVersion() == 1) {
     header.append(";Version=1");
     if (cookie.getComment() != null) {
       header.append(";Comment=\"" + cookie.getComment() + "\"");
     }
     if (cookie.getMaxAge() > -1) {
       header.append(";Max-Age=" + cookie.getMaxAge());
     }
   } else {
     if (cookie.getMaxAge() > -1) {
       Date now = new Date();
       now.setTime(now.getTime() + (1000L * cookie.getMaxAge()));
       header.append(";Expires=" + HttpOnlyResponseWrapper.cookieFormat.format(now));
     }
   }
  
   if (cookie.getDomain() != null) {
     header.append(";Domain=" + cookie.getDomain());
   }
   if (cookie.getPath() != null) {
     header.append(";Path=" + cookie.getPath());
   }
   if (cookie.getSecure()) {
     header.append(";Secure");
   }
   header.append(";httpOnly");
   addHeader("Set-Cookie", header.toString());
 }
}

public class HttpOnlyFilter implements Filter {
 private FilterConfig config;

 @Override
 public void destroy() {
   this.config = null;
 }

 @Override
 public void doFilter(ServletRequest req, ServletResponse res,
     FilterChain chain) throws IOException, ServletException {
   System.out.println("Got here");
  
   HttpOnlyResponseWrapper hres = new HttpOnlyResponseWrapper((HttpServletResponse)res);
   chain.doFilter(req, hres);
 }

 public FilterConfig getFilterConfig() {
   return this.config;
 }

 @Override
 public void init(FilterConfig config) throws ServletException {
   this.config = config;
 }
}

Jeremiah Grossman: HTTP Response Splitting Revelations

Jeremiah Grossman has released some additional information on pervasiveness and severity of HTTP Response Splitting (HRS¹, not to be confused with HRS² which is HTTP Request Smuggling).
While the recommendations are spot-on (input validation, output filtering), I'd say they're a bit incomplete:
1) Input validation should always be whitelist. The recommendation there was to remove carriage returns or linefeeds. My recommendation is don't allow it if it doesn't fit the model you're looking for.
2) (This is the more important). In my experience, it's almost never necessary to return a user's direct input as a header. The most common cases where I see Response Splitting/Header Injection are when a user preference cookie is set (what's your favorite background color?) or in redirects. If it's a user preference, that needs to be kept in the user's session, not as a cookie in the user's browser. And if it's a redirect, if you can exploit an HRS by a redirect, you've almost certainly got an open redirect issue as well.

So there's my amendment:
1) Business rule - whitelist input validation
2) Proper output filtering (some of the Java API's for writing cookies and headers will even throw exceptions if the output isn't properly encoded, but the behavior is inconsistent - some throw exceptions, others don't throw exceptions but encode for you, some do none of the above).
3) Look at the engineering - is the information you're putting in a cookie really necessary to put in a cookie? Does it make more sense to put in the session?

Going off the Rails on a Groovy Train - Part 2

Well, in Part 1, I promised to explain why I went back to Groovy after ignoring it for so long. But to explain that would be to explain a whole bunch of other stuff. While not completely security related, it might be interesting for developers to see the thought process that makes a wishy-washy person like me jump ships on languages.

First, I'm not a religious zealot about languages. To me, languages are just tools, and learning a new language is a matter of learning syntax, and how to find the libraries I need to do the job.

For the longest time, I was a Perl coder - er....line noise coder. My code was write-only. In Java parlance, write once, ignore forevermore. The two major benefits of perl for me were availability of libraries and documentation for those libraries (owing to perl's maturity), and syntactical sugar. (Of course, I didn't know the benefits of the second until I couldn't use it anymore).

I used perl a lot. But for pen testing, I kept coming into situations where it wouldn't cut it anymore - almost always due to finicky proxy requirements. I could do SSL, I could do proxy, I could do NTLM autentication, but I always seemed to have issues dealing with combinations of the three. And it seemed that many, many pen tests had something come up that required some scripting and that the tools out there simply couldn't handle.

So for one assessment I tried out Commons HttpClient with Beanshell. I had been using Java and Struts for web development for awhile, so the transition to Beanshell was natural. And HttpClient did exactly what I needed when I couldn't seem to make Perl do it. (My apologies to the perl people for giving up so easily, but I had an immediate need. And my apologies to the Python people because I prolly never gave Python a fair shake).

So I stuck with beanshell and HttpClient for awhile. But beanshell never really made me happy. While a dynamically-typed java was nicer, it never got to feel like a scripting language because I still didn't have a lot of syntactical sugar. My colleagues jumped on the Ruby bandwagon quickly, and I jumped off as quickly because the two things I needed to be able to do with it were difficult to find libraries for. And in fact, did a lot of assessment work where I'd use beanshell to do the http plumbing, and perl to do the parsing because parsing HTML in Java is so un-natural - possible, but not natural.

I'm not sure why I gave up on groovy so long ago. I'm not sure why that one afternoon I dusted off beanshell instead of dusting off groovy, but I wish I had picked groovy then. A couple of weeks ago, I decided to look at groovy again when I had had enough with beanshell, and I was floored. It was like magic. All the library maturity of Java (I still get to use Jakarta stuff) with all the syntactical sugar of perl and all the readability of Ruby. And that there are builders for almost anything hierarchical in nature.

Going off the Rails on a Groovy Train - Part 1

No, this was not an entry in the "Blog Post with the Most Puns or References of Ozzy Songs" contest.

Many of my colleagues are (as is a lot of the web development world) really high on Ruby on Rails. And I can understand why. Really cool Web 2.0 sounding statements like "convention over configuration" or DRY (Don't Repeat Yourself - which I just did by telling you what the acronym meant right after the acronym) are not just words, they're baked in to the way Rails operates. Rails makes the every day of web application development uber-easy. And heck, Ruby has closures which almost any legitimate scripting language really needs.

But I've always had some problems with Rails. Some of these are admittedly probably based on my fear of change, but the four most substantial ones are:

• Lack of libraries: to really make your life easier, a good scripting language or framework needs to have lots of libraries to do what you need. Ruby does have lots of libraries. But when I started trying to learn it, I had two needs, and I could not find libraries for either of the two. My Rails-uber-guru colleagues couldn't find them either. That doesn't mean they don't exist, but in Perl - you go to CPAN. In PHP, you go to the centrail PEAR repo. In Java, if it's not there already, Apache already wrote it. In Ruby, it might be on Rubyforge. Or there might be thirty of them hosted on various people's blogs - no telling which ones are bulletproof. This will get better with time.
• Lack of documentation: which is probably a legitimate complaint for anything FLOSS. And this goes hand-in-hand with the lack of libraries (or lack of a couple of de facto standard repo's). With Java, you know the doc is good.
• Inability to leverage existing work: For new applications, Rails is fantastic. But because Rails runs on Ruby, there's not (yet) a way for me to leverage my existing work in existing VM's without turning my existing work into web services (shame on me for not doing it right the first time - but did you?)
• Key exposure: Part of the convention over configuration theme turns into key exposure. URL's say things like http://server/blog/post/edit/832 - where 832 is the primary key field of the post I'm editing. Now, key exposure is not that big of a deal if 1) the key by itself has no meaning, which if you follow convention, it doesn't, 2) there's no SQL injection, which ActionRecord deals with nicely, and 3) you do permission checking for that object on every action - but that check is up to you.

That being said, Ruby on Rails is a really big and good step in the right direction. However, the first three items are killer for me - it doesn't matter if I can "scaffold" a really complex schema in 8.3 seconds if I'm going to spend 8 weeks writing my own XSL translator or re-inventing the toaster oven.

In the next couple of posts, I'll explain why I looked (back) into Groovy and why I'm (currently) geeking on Grails.

Do We Really Need a Security Industry

When somebody posts something critical of what I've written, I'm careful to go back and read my own words. And I frequently find that I'm flat out wrong. And in other cases, such as this one, I write before I think (similar to implementing before you engineer) which ends in me looking very unprofessional.

RSnake was (rightly) defending himself after I made a flippant remark about his warming up in one circumstance to WAF's. And my remark neither fully explained RSnake's decision, nor gave a complete picture of the depth of the decision in one circumstance. RSnake gave a great deal of thought to how WAF's can be beneficial, and certainly did not say they were some sort of a silver bullet solution to all security failures.

Here's my take on WAF's. Like RSnake said about Schneier, I wish it were a perfect world. I wish programmers didn't make mistakes. Even earlier, I wish software engineers thought more like civil engineers. I wish the really poorly-conceived models we have, and poorly-coded applications we have, and all the insecure frameworks we have could just be chucked and re-conceived, re-coded, and re-engineered. But as has been pointed out to me many times, I have my head in the clouds, and not grounded in reality.

That being said, if you are able to go back to square one on an application, or if you're in the infancy of engineering something, it is far more effective and cost-effective to design things right the first time. If you have a real need for WAF's in the future, then you need to go back and look at your processes to see how you can do things better on the next iteration. WAF's are great as a safety net, but will never help your programmers and engineers do their job better, but they may have the adverse effect - by covering up for flaws made by programmers, they may train the programmers to not do the right thing and to completely rely upon the WAF (or the hacking team, or the security scanner) to catch their failures.

WAF's can be fantastic in those circumstances like RSnake spoke about - where re-engineering is a complete non-option, or when the application is never going through other iterations and will soon be retired. And I'm not even supposing that you shouldn't always use them - but when starting from scratch, never rely on them to provide security you should have baked in yourself. WAF's don't know your application the way you do, and in order to make them know the application the way you do requires you to write logic into the WAF that you simply should have written into the application.

Now, as far as RSnake is concerned, I value his opinion more highly than my own. He's the one who's been in (and out) of the industry far longer than I have. He's the one who's asked to give lots and lots and lots of presentations, not me. The masses respect is opinion much further than mine (and for once, in this case, I think the masses are right). I ground almost everything I write in the philosophy of how you should write things if you're writing them new. RSnake is firmly grounded in reality. If he says something is busted, you'd better read what it is, and figure out if yours is too. If you're writing from scratch, or doing a complete re-write, I highly encourage you to look here and see if there are good practices you can employ to make your application better from the beginning.

RSnake, my apologies for making such a statement without fleshing out the thought. In the event we ever get to meet up, I'll buy you a pop - but seeing as how you're in Texas now, it's all called Coke regardless of what brand or flavor it is. "You want a Coke?" "Sure!" "What kind?" "Sprite."

False Positives vs. Non-Exploitables

Another professional and I sat down to coffee a few weeks ago, and we got around to a conversation we seem to have often. I feel almost ashamed to publish a post on it when there's no "other side of the argument" here to defend itself. So please bear with me as I try to do both arguments justice (and tell you why my side is right).

Consumers of security tools have long asked that the tool reduce the number of false positives. And this is certainly understandable - it's very difficult to know what to fix when the list of things that are broken includes so many things that aren't broken. All areas of security try to sell you on their perfect false positive to false negative ratio. Theoretically, the two are on separate paths that at some point intersect - as the sensitivity of the tool is increased, false positives go up, false negatives go down. As the sensitivity is relaxes, false positives go down, false negatives go up. The theoretical ideal of the two is where the two are equal - this would be the lowest sum total of the two.

Tool vendors have to make their customers happy. And false positive to false negative ratio is one metric that all vendors use to prove the value of their tool. None of the tools out there use the same taxonomy, result format, reporting, etc., so the only apples-to-apples comparison we have is false positive to false negative ratio.

Now, the tool vendors work very hard to get the false positives to decrease, and depending on the type of tool, they do it varying ways. Application firewalls may use anomaly detection rather than flagging everything with known-bad data. Static analysis tools build a call tree to see if the data is sanitized anywhere in the process, and if so, it's not reported. Dynamic analysis tools don't report "evidence of", they report real exploits.

This is both good and bad. The good side is that because false positives are low, when something is reported, you can be pretty sure it needs to be fixed. When a static analyzer finds an injection flaw, you know that the data goes soup to nuts without being properly checked or encoded. When a pen testing tool finds something you know it didn't find those pesky points where you got back an angle bracket, but just couldn't get a real injection attack.

However, I think that false positives have been confused with non-exploitables. When pen-testing, those pesky points where you get back a real angle bracket, but can't get a real injection to work, although there's no injection, you have sufficient proof that encoding is not taking place. And in static analysis, if you see a place where data goes to a "presentation layert" (database call, screen, wire, LDAP server, command shell, etc.) without being properly encoded, you have a real flaw, even though it's not exploitable.

There are two primary reasons that not reporting non-exploitables is dangerous:

1. Although the point is not exploitable today, there's a false safety net keeping it from being exploitable - a string length restriction that could change tomorrow, or an input policy that happens to be checked today, but when the same input is accepted from another source tomorrow, the same checks don't take place.
2. When asked to fix problems, developers must take the results of the test to make their decisions on what to fix. If output is not encoded in two places, and only one is reported, how will the developer learn what to fix by output encoding? They'll learn to fix the ones the tool reports. So how so developers learn to code correctly? They don't - they learn to code the way they always do, then count on the tool to correct them later. Even if it's not exploitable, non output-filtered code should be flagged (possibly rated differently in severity).

So for the security practitioners, do your developers a favor and document those "uncomfortable places" where you couldn't get a really good exploit to work, but you know there's something not behaving properly. When you're doing a manual code review, flag all of those places where data goes to a presentation layer without actual encoding.

And developers, do yourselves a huge favor and when a report comes back with flaws, demand that whoever made the report (your tool, your security team, your peer reviewer), explain to you why it's a problem. Don't simply fix that issue. Discover how you fixed it, and how you can apply the same fix to other locations that weren't reported to you. And more importantly, make that fix a part of your programming habits, so that you just write it that way in the future.

Struts, I never knew thee

It's certainly not a complete transformation, yet, but in small projects, I think I'm going to give up on Struts Action, and am going to take Stripes for a spin. Once I finally got my head around all the configuration (or at least enough configuration to get an application operational), I really began to love Struts, but there was always something very inelegant about it.
Now, all my Ruby on Rails fanboy co-workers make fun of Stripes because in Java, it's probably too little, too late. But to me, it's a close-to perfect balance between Struts (where you must understand everything that's going on) and Rails (where you don't know squat about what's going on). I have a problem with not knowing what's going on, as in Rails. And I know that if you spend enough time, you can figure out what Rails is doing. But it's ridiculously simple to do the ridiculously simple in Rails, but if you need to customize that in the least (say, remove key exposure), then you're going to spend as much time re-implementing things, in which case you save yourself no development time, and at a substantial cost (it's basically CGI, so long-running processes have to be fork()'ed and detached). And with Struts, knowing what's going on comes at a huge cost - you have a bazillion configuration files, and as soon as you use the Struts tag libraries, you have to make all the components (an action, a form bean, and the JSP) all at once.
So my task for this week is to get a nice-looking base ActionBean class for Stripes that I can use with utilities necessary to generate and check action tokens to deal with XSRF and Javascript Hijacking. Then on to adding Prototype to some mock-up apps around the house.
And if you can't tell from this post, I'm really slow about adopting new frameworks.

Should We Trust Frameworks?

In light of Fortify's recent study of FLOSS libraries and frameworks for Ajax with regards to Javascript Hijacking, I thought I would provide some help in making decisions on whether to use a framework/library/API, or to roll your own for what it is you're doing.

Obviously, it would be foolish of me to have a "sky is falling" mentality about the findings, but there are some common-sense things to keep in mind when selecting an API or writing a new API for what you need to do.

• New is not always better. It's just new. This applies to API's and technologies both. For example, when some FLOSS project releases a brand spankin' new Ajax library, that doesn't mean it's better than everything else out there.
  
• Do you need a new technology? Is Ajax really going to substantially improve your user experience? Or are there other features that should be implemented first? Whatever the new technology is that you hope to implement with the API, evaluate your need for the technology in general before you even go to selecting a tool provider.
• Is the provider a trusted source? This is becoming harder to deal with as really rich, interactive web UI's are becoming the norm. Several years ago, there was a shift to making large improvements for the sake of developers, and MVC was going mainstream in webapps. After a few years, a handful of really good MVC frameworks emerged. Can you wait that few years for the really good new framework to emerge?
• I don't have a metric for this, but is there a "number of man-hours invested into development" appropriate for the function the library provides? For example, are you going to pick an API by a guy who spends a couple of hours on the weekends developing the tool, or something where there's a really active development team spending a total number of hours far exceeding the one guy? More is not necessarily better, but in the FLOSS community, really active projects generally get a little more momentum in getting things fixed.
• Is the API/toolset extensible? This is a huge one for me. I want to like Ruby on Rails, but I have a real problem with key exposure. And I have a real, real problem with not being able to work my way to the internals to make Rails work with a layer of indirection without having to write all my apps to use it - end result, it's just as quick for me to write my apps in a "slow" framework where I can change as necessary.
• Has the provider responded to findings in the past? Do they fix security things? Or do they ignore them and possibly provide documentation so you can fix it yourself?
• Will the API save you enough time and give you enough flexibility to outweigh the benefits of writing it yourself? I can't phrase this in the right way strongly enough. Fortify found problems with a lot of Ajax libraries. But they are far more likely (the active ones, anyhow) to fix the problem and fix it right than you are, unless you're one of them. Not an insult to you, but if you're really good at writing applications, try to spend your time doing that, rather than writing your own frameworks.

The question on the last point probably came out wrong. If you even think for a moment that you can write the API yourself, please reconsider and look into some of the more mature API's and frameworks out there. I think all of us who do security assessment have seen too many opportunities to use good API's go by the wayside and be replaced with home-grown API's that are really deficient. The two things in the past that I still see come to surface are home-grown crypto API's (really bad idea), and home-grown MVC that almost always have directory traversal, forced browsing, or arbitrary object instantiation issues.

So, even though Fortify found some problems with the frameworks out there, please consider using one of those. But before you consider using one of those, consider whether it's really time to implement that shiny new technology at all. You might have better functionality that you can provide without committing to something that's really immature right now.

It's a real patent

In an earlier post, I mentioned that somebody managed to get a patent for the doubly-linked list. A friend told me it was a hoax, so I did a little research. Turns out, it was not a hoax - Ming-Jen Wang of LSI was awarded patent number 7028023. The friend who told me it was a hoax is no longer a friend.

On the other hand, it's not simply the doubly-linked list that was patented. It was a tertiary list of pointers for indexing on top of the doubly-linked list. That is, a doubly linked list allows you to iterate through the items forward and backwards by some key field. Wang's patent is on adding another key field that you can iterate through the list by something other than the key field. So it appears one of two things is going on here: 1) I totally don't understand the patent (entirely possible - feel free to correct me if that's the case), or 2) a patent was awarded to an incremental improvement of an existing algorithm by implementing the same algorithm a second time. Like making a brick wall stronger by using a second layer of bricks.

Patent Nonsense

Admittedly, I've not read the entire patent, but this guy managed to get a patent for the linked list. If you read only the Abstract, it looks like the patent would cover using a list of pointers to point to items in a collection - not the items in the collection containing pointers to the next item. Reading the details, though, shows this not to be the case (phew! Was thinking that even Blaise Pascal wouldn't be immune to this one!)

It looks like (only when you read the detail) that the patent would truly only cover doubly-linked (or n-linked) lists, not singularly-linked lists. However, this is still a really broad patent of a data structure. I told a buddy who is a transportation engineer that this would be akin to patenting the wheel. Not the tire. Not a particular type of wheel. The wheel.

Living in a capitalist society, the one thing that bothers me the most about the patent office is not the patents they allow through. It's that the patent office has a monopoly on the patent process. See, as security professionals, if we decide that a standard of measurement is inadequate, we can invent a new standard of measurement, get a bunch of other security professionals on board, and the standard is effectively changed. What we need is a new patent office. But if we were to start a new patent office, and everybody bought into it, it still wouldn't be the patent office.