PHPIDS » Web Application Security 2.0

Today we finished the PHPIDS white-paper which was created as a reaction on the CFP for OWASP Europe Conference 2008 in Belgium.

The paper includes general project information, installation tips and detailed insights into the attack detection work flow. Also the paper features an explanation of the PHPIDS Centrifuge as well as some best practices on how to work with the impact and discussion about server performance issues.

Comments on the document are highly appreciated – so feel free to contact us if questions pop up. We hope this paper sheds some light on the black majick of the attack detection mechanisms the PHPIDS features which separate it from comparable monitoring solutions.

You can find the paper here.

Edit 2008-04-30:

Here’s the most recent version – great thanks to David Lindsay for helping out with proof reading and wording issues.

Only half a month after the last release we present the new PHPIDS 0.4.6. This time we did lots of optimizations on the generic attack detection and the PHPIDS Centrifuge. There is a pretty new way to detect vectors which are not caught by the rules and as far as we heard we returned some of the headaches our testers gave to us before – thanks again to David Lindsay, Gareth Heyes and Johannes Dahse for their great work.

The rules were optimized as usual and again – they haven’t grown but become even smaller for better performance. Altogether the rules decreased their size by 937 bytes. The converter was optimized too and many smaller bugs were fixed.

You can find the fresh packages here as usual. Again – no API changes so updating should work like a charm.

Also we have continued working on our sister project – the CSRFx. Now this system is even more capable in dealing with invalid markup to protect and AJAX requests. Also JSON wrapped markup can now be secured with the token cloud of the CSRFx so maybe you like to check out the sources and give it try.

We appreciate your feedback and if you happen to have any problems during installation or usage feel free to ask us in our forum.

After the pretty successful Christmas release we now present PHPIDS 0.4.5. It brings a lot of enhancements in vector detection. We worked over the rules and especially the converter and due to the great help of David, Gareth, Johannes and tx many bugs were found and fixed. The exploits and filter circumventions they found were awesome as usual and got our team surprised a lot. JavaScript is a hell of a language – and so is SQL…

We also did some improvements to the PHPIDS Centrifuge. We now have – supporting the main Centrifuge core – an additional layer to detect attacks based on character ratio. Take a peek at the code if you wish to know more details.

The API hasn’t changed in this release so patching would be definitely no problem as usual. We hope you like the new release and grab you package here! Also we heard some bird twitter about a new WPIDS release some when this week – stay tuned!

Just in time before the holidays we are proud to release PHPIDS 0.4.4. After several weeks of testing by the group we populated the new release with small but important features and optimizations. We added support for detection and translation of JavaScript Unicode – undetected vectors like \0061ert(1) Gareth Heyes discovered now belong to the past. Also we optimized the rules to catch the latest concatenation and code injection vectors crafted by thornmaker and tx.

The centrifuge was optimized a little bit more and here and there we heard about vectors that were exclusively detected by this mechanism – so yes, the concept works. Furthermore we discovered several minor bottle necks when dealing with very large incoming strings which of course were removed too for better scalability and performance.

We also increased the code quality – the PHPIDS is now completely coded in PEAR valid PHP constantly monitored by the PEAR package PHP_CodeSniffer. The test coverage is higher than 95% and we also tweaked the generated documentation for better understanding.

We hope as usual that you like the new release as much as we do and wish you a very happy and relaxing holiday. See you next year!

Today we proudly release PHPIDS 0.4.3. This time we invested all spare time we had the last weeks on enhancing the converter and the rules. That means way better intrusion detection and even fewer false alerts then with the last release.

Thanks to the great help from Johannes Dahse we managed to tweak the rules to catch way more SQL Injections – especially the super short ones for authentication bypass and information disclosure. And – we didn’t believe it ourselves – SirDarckCat and Gareth Heyes even found some new XSS vectors slipping through the rules. We also optimized the converter against several evil Unicode characters and other possibilities to obfuscate payload.

Furthermore we did some more testing and optimization on the PHPIDS centrifuge. After several weeks of high traffic beta testing we agreed to remove the ‘beta’-label from this module too. Be sure to grab the files from here as soon as possible

The coming releases will head straight towards 0.5 – the usability and scalability release. We hope you are looking forward for this one as much as we do – and enjoy PHPIDS 0.4.3.

Today we are talking to Reiners who helped us enhancing the SQL Injection detection rules. Thanks to his outstanding work we were able to identify lots of bugs in the rules and make the PHPIDS a lot better in SQL Injection detection that we ever thought it could be.

Q: Please tell us a little bit about yourself?

My name is Johannes Dahse and I am studying “IT-Security” at the Ruhr University Bochum in Germany. Beside my studies I read a lot about websecurity and experiment with it or I write some codes for smaller projects. I also like to work out, and hang out with friends and grab some beers.

Q: During the last weeks we happened to learn to know you as a top notch SQL Injection expert – how come?

It started with learning PHP and MySQL about 4 years ago. Back then, I was already interested in security in general and did a lot of research. While participating at the last CIPHER (a Capture The Flag-style wargame) I noticed that my SQLi knowledge was a bit rusty and started to do more research on it which leads me to PHP-IDS. I learned a lot during the challenge to trick the filters and had a lot of fun.

Q: XSS vs. SQLI can you compare them? If yes, whose impact is bigger?

That is an interesting question. Generally I would say SQLi is more dangerous because it is a server-side problem and can lead to a full takeover. But it depends on what the attacker wants to do, the DBMS and its settings of course. And there may be a lot of scenarios where XSS as client-side attack is way more effective to reach your goals.
You shouldn’t have one of those holes in your webapp anyway, but I’d rather like to know a XSS hole in my app than a SQLi

Q: WebAppSec in five years – any prognoses?

I think WebAppSec is getting more and more important. The amount of web-applications is growing, however, most of their developers tend to ignore web security. Additionally, many people release their personal information at the internet and therefore, the security for these personal data will play a major role.

Q: Whom would you like to invite for dinner and why

Haha, I can think of a couple of lovely ladies I’d like to meet but I guess you are asking towards webappsec. Well, since I really enjoy reading Ronalds blog (www.0×000000.com) I think it would be really interesting having a chat with him. He has some really interesting posts I’d like to talk more about.

Q: Thanks for the interview!

After a pretty long time without releases we finally present the PHPIDS 0.4.2 which ships a long awaited and heavily demanded feature – absolute paths in the Config.ini. Besides this major change we have done tons of improvements to the rules – especially to the rules to detect SQL Injection patterns.

Furthermore we heavily reduced false alerts again – PHPIDS 0.4.2 is the first release that comes without any false alerts from the false alert DB which you guys kindly helped filling with various input and suggestions. Also the PHPIDS is now capable of detection XXE attacks and basic LDAP injections. We also tweaked the converter and the almighty PHPIDS Centrifuge to ease the preparation of possible attack vectors and make the detection process even faster than before.

Our test suite has meanwhile grown to 75 test cases and covers almost any recent detection bypass to make sure no older attacks slip through due to rule changes.

The next releases will aim to ease implementation and usability of the PHPIDS and most importantly to make it more scalable on very large environments – any suggestions or feature proposals are very welcome as usual.

We hope you like the fresh release and like to thank all the people helping with testing and enhancing the PHPIDS – have fun!

Today we are talking to Giorgio Maone who helped us several times improving the PHPIDS filters and converter with elegant XSS vectors. He was the first to break the filter after the XSS contest begun and some weeks before he generated real headaches on our sides by exploiting the name-trick in multiple ways. Ah – and not to forget – he has a cool blog too

Q: Please tell us a little bit about yourself!

A: I’m a senior software developer and CTO at InformAction, an Italy-based IT consulting firm I co-founded in 1998. I love slow food, martial arts, elegant code, listening good jazz and playing bad jazz. Some competitors keep accusing me of being the evil mind behind NoScript, but I plead not guilty: it’s just a setup to scare way Web 2.0+ investors from my company. Ajax, Comet, Widgets and Mashups FTW!

Q: You have a pretty impressive vita as developer and security expert – how did it all start?

A: My dad took home a Commodore 64 almost 25 years ago, and I immediately started hacking it passionately in a few weeks. 50KB RAM and a supernaturally fast 1MHz CPU can do wonders at carving your notions of “code bloat” and “optimization” in stone. My Web debut has been with Mosaic, an Amiga porting of the venerable NCSA browser with no JavaScript interpreter… did you say imprinting?! When I had my first RDBMS experience, injections were almost impossible: SQL statements were embedded in Cobol programs through a pre-compiler and parameters were safely bound to variables. I’d dare say we saw an involution, through Visual Basic to Web scripting, and only recently the
mainstream (the PHP/MySQL crowd) is red discovering prepared statements and parameter binding, which have always been there, e.g. in JDBC.

Q: Firefox 3 will bring us… please complete the sentence

A: It depends on who’s “us”.

• Users: brand new bookmarking, tagging and rating system (Places), malware blocking (Google-powered blacklist, no less!), disconnected web applications, cross-session resumable downloads, UI to disable plugins
• Chrome developers: simplified JavaScript API (FUEL), site-specific preferences, reliable SQL storage (it’s already here, but now we can stop worrying about legacy compatibility)
• Content developers: offline persistence, better SVG, CSS3 and ACID test compatibility, many HTML5 features
• Black hat folks: offline persistence, better SVG, CSS3 and ACID test compatibility, many HTML5 features
• White hat folks: lots of fun!

Q: What would you forbid if you had the chance

A:

• Stupidity
• Corporate greed
• Globalized exploitation (and no, I don’t mean a PHP attack involving superglobals)

Q: The PHPIDS is… please complete the sentence.

A: The Sudoku killer! The new Rubik cube!! Hours and hours of unlimited fun!!! No, really, I believe it’s a very useful project and a great first line defense against generalized attacks, e.g. those targeting popular CMS packages like Joomla or WordPress whose quality is not entirely under your control. Blacklist-based filters cannot replace good coding practices, and a really motivated and skilled enough attacker can always find a way around sooner or later, but if your sysadmin is attentive and your developers know their stuff, PHPIDS is surely a precious tool to detect suspicious activity and harden your walls before it’s too late.

Q: Thanks a lot for the interview, Giorgio!

It’s not that long ago that we released PHPIDS 0.4 but nevertheless this release brings some very interesting new features you may like. To make things short – here’s a list:

• Getter and setter methods for the config array
• Several completely reworked rules – thanks to the heavy testing in the PHPIDS Group
• Way less false alerts, more true ones with fewer rules
• A pretty new feature called ‘PHPIDS Centrifuge BETA’
• More tests and better coverage
• Simplification of the settings for the Caching features
• Several minor bug fixes

The PHPIDS Centrifuge BETA is basically a new method in the Converter class which is able to detect unknown attack patterns not covered by the rules yet. This is performed via acting on incoming and suspicious string like a centrifuge- stripping all uninteresting characters, normalizing some others and finally having a result that is very short and reflects around 75% to 85% of the attack vectors we tried during testing – and we tried a bunch of. We are still testing and enhancing that feature so the minimum amount of chars for a string to be mangled by the Centrifuge is set to 80. Thanks Martin and Gareth for your help with the testing.

Furthermore many users requested the possibility to have a setter for the config array inside the Init object. This is also included in this release and opens the 0.5 sprint we named ‘Usability & Scalability’.

We hope as always that you like this release and thank you for your support. You will find the download packages here.

This time we talk to a guy who approached in the group a while ago and submitted tremendously obfuscated JavaScript vectors and pointed out important flaws in the recent rule revisions. Since he came up with vectors containing characters like ‘ä’ and ‘ö’ we were pretty sure is from Europe – now we know more. Here’s what xorrer has to say:

Q: Please tell us a little bit about yourself!

A: I’m a software engineering student at the Vienna University of Technology. Currently I’m working for a small company on their web shop. I like to listen to all kinds of music, my all time favorites are The Doors.

Q: How did you come to webappsec

A: To be honest, a few weeks ago I wasn’t at all into webappsec. I guess I was reading/researching some stuff for work when I came across some RSS-feed about XSS. After a little research, a few other feeds (e.g RSnake’s XSS cheat sheet) I was into it. Since then I read a lot about PHP Security, SQLI, XSS, you name it. And with this entry I got to the PHPIDS project.

Q: What do you think about JavaScript – especially at the moment

A: That’s a funny question, as before writing some vectors for PHPIDS I never really did anything in JavaScript. Now after some fiddling around, while testing PHPIDS, I think that JavaScript is the ultimate web security nightmare. As Gareth already stated “Javascript: The ultimate hacking tool“. There is just way too much you can do with it and nowadays you can’t even turn it off in your browser, as there are many sites out there which won’t work without (Ajax, jQuery, Dojo, …).

Q: You’re hat is… please complete the sentence!

A: White. Responsible disclosure is the way to go. There are just too many script kiddies and other people out there for full disclosure.

Q: How would you imagine webappsec in five years?

A: I guess that for some years to come the situation will remain basically the same. The protections are just inferior to the attacks. And this won’t change so easily, as long as no company really gives (or even knows they should give) a thing about web security. Everyday there gets launched a new service and a new technology and in that rate there will be new un-thought of vulnerabilities and new attacks waiting.

Q: The PHPIDS is… please complete the sentence!

A: A good weapon for the first row of defense. It’s build by people who know what they are doing and tested by experts on the field of web application security.

Q: Thanks a lot for the interview!