HTML Unit

In a previous post, I mentioned that HTML Unit uses Commons HTTPClient, but wasn't certain if the HTTPClient objects were exposed. As it turns out, the HttpState is exposed - client.webConnection.state will get you there. There are warnings all through the javadoc that this may be gone tomorrow, use at your own risk, but for the time being, that's how you can get through authenticated proxies.

For example:

wc = new WebClient(BrowserVersion.MOZILLA_1_0, '127.0.0.1', 8080)
state = wc.webConnection.state
state.setProxyCredentials(AuthScope.ANY,
 new UsernamePasswordCredentials('user','password))

Strangely, there's not a way to set the proxy host and port after the client has been constructed (as far as I can tell), meaning you have to specify a browser version (which is just as well - you're mimicking a browser, why not fake the UserAgent as well?)

MITM Proxies - and other tools

RSnake was letting us all know that Portswigger is taking requests for new features for Burp. Burp proxy is one of a handful of tools that you must have in your toolbelt for doing manual assessments. Funny how I've never shared what I have in my manual assessment toolbelt:

• MITM proxy - my standard is Paros, but also keep installs of Burp and WebScarab. What you need first and foremost is a proxy that works. Each of them have pros and cons as far as being able to reliably connect under various circumstances. If you're consulting or dealing with vendor apps, you'll probably use all of them at different times to deal with different proxy configurations (I mean corporate proxy, not MITM proxy), host authentication methods, SSL, etc. If the site is SSL and has applets, you'll need one that you can easily modify (this is why I use Paros) so you can change which keystore it uses and make a hacked self-signed cert. Next time I have to do something like this, I'll write a blog post about it. It's not hard, but it's not obvious, either.
• Scripting language - I prefer groovy now. Some assessors I know never once write a single line of script. I can't do a single assessment without writing some amount of it. Some people prefer Python, and I just stopped using Perl a year or so ago. I don't know if Ruby has the libraries it needs to make it through various authentication and proxy schemes yet, but I'm sure it will be soon. I switched to Java-based languages because of all the different hoops I have to go through for assessments - Commons HTTPClient can get me there, no matter the circumstances. The Java API's for HTML parsing are less than snappy, but adequate (maybe another post on that), but lately I've started using NekoHTML which balances HTML tags and such to turn it into something you can use XML parsing on, and this weekend finally got around to playing with HTMLUnit (which uses Commons HTTPClient and NekoHTML) - and it makes a lot of tasks a lot easier. It even parses and executes JavaScript, but I've had issues with a few sites (sites with Google Ads in particular), with the Javascript parsing, so I just disable it there.
• A good text editor. A "good" text editor is critical just because my editor ends up being my "landing zone" for everything. So you want your text editor to have some of the following features: regex search and replace (absolute must), macro ability (you never know when you need to just cobble something together), tabbed interface, column editing (you'd be surprised how many times I use just this one feature in an assessment). Right now, I'm pretty well stuck with jEdit, but prior to that I used UltraEdit exclusively. I tend to like JEdit now simply because of one feature - Beanshell is built in. This is handy in two places - the first is for encoding and decoding - I've written a handful of macros for doing encoding and decoding (hex->binary, binary->hex, digests, HTML encoding, URL encoding, Base-64, etc.) - but your proxy may also provide this - just nice not to have to switch. And you can make the results of a search and make the replacement the evaluation of a beanshell script, which is handy when dealing with obfuscated code.
• Firefox with at least LiveHTTPHeaders, SwitchProxy, User Agent Switcher, and Firebug. What I ended up doing is using a hacked up Portable Firefox for doing assessments, and my primary day-to-day firefox doesn't include all the extensions.
• The Gimp. May seem live overkill when you just need screenshots, but you also needs something that can do a really good Gaussian Blur or similar for redacting. With a normal desktop paintbrush, redacting means putting an ugly black bar in front, which for a finalized report doesn't look that great. You want something you can annotate with and make pretty red circles.
• I've thought about desktop recording software, and I've seen some really nice ones with voice recording and graphical annotations, but you can't put a video in a printed document, so it's not something I've used a great deal.
• The XSS Cheat Sheet. I wish there were SQL Injection (or LDAP Injection, or name that Injection) equivalent for all the various RDMBS out there, but doing a little research on the specific RDBMS and a good set of encoders will get you far.
• There's just no substitute for being able to figure out what's going on in the backend. The more opportunity you have to turn your blackbox test into a graybox test, the better your chances of getting a really good exploit. There's not a tool for that, but just know that it's hard to train somebody to be a truly 1337 hacker who doesn't understand a lot about the systems it runs on - application level or OS level. So that being said, there's no substitute for research.

What tools did I miss (besides Telnet?)