PHPIDS » Web Application Security 2.0

This weekend I got feedback from the OWASP Crew from Belgium. The talk evolving around the PHPIDS Whitepaper was accepted and found a slot in the time line of the OWASP AppSec Europe 2008.

Don’t miss this event if you want to meet team members of the PHPIDS in persona as well as outstanding security experts like pdp, Ivan Ristic, Martin Johns and many others.

The talk will range from 14:40 to 15:20 in the second track at 21st of May 2008. Main topics are the PHPIDS, how it works, what the major benefits and possible drawbacks are and of course how the black-majickish Centrifuge works and how other tools can utilize its logic. We will publish the presentation and if available a video of the talk for all who unfortunately can’t participate.

We are glad to announce the freshest release of the PHPIDS. As you might have expected we did a lot of work optimizing the converter and the centrifuge again. Also the rules were improved slightly to catch several sophisticated SQL Injection vectors Johannes Dahse submitted. Again we have to thank David Lindsay, Gareth Heyes and others for their great work. The system wouldn’t even be as half as good without their contributions and intense testing.

The PHPIDS now performs way better when dealing with UTF7 XSS and especially data URIs with mixed encoding. Gareth and his outstanding Hackvertor managed to create some weird but sophisticated examples of how data URIs can be obfuscated to the max. Don’t forget to check out his amazing tool.

The PHPIDS now also ’speaks’ Base64 - so no vector obfuscation with this encoding anymore, bad guys! The count of false alerts has decreased amazingly with the new rules so if an incoming string was detected as suspicious by the PHPIDS you can almost be 99% sure that it was an intrusion attempt.

We’d also like to thank the community from our forum for the help on optimizing the system and adding improvements here and there. Be sure to grab the latest packages here - again no API changes by the way so patching will work without any problems.

Today we finished the PHPIDS white-paper which was created as a reaction on the CFP for OWASP Europe Conference 2008 in Belgium.

The paper includes general project information, installation tips and detailed insights into the attack detection work flow. Also the paper features an explanation of the PHPIDS Centrifuge as well as some best practices on how to work with the impact and discussion about server performance issues.

Comments on the document are highly appreciated - so feel free to contact us if questions pop up. We hope this paper sheds some light on the black majick of the attack detection mechanisms the PHPIDS features which separate it from comparable monitoring solutions.

You can find the paper here.

Edit 2008-04-30:

Here’s the most recent version - great thanks to David Lindsay for helping out with proof reading and wording issues.

Only half a month after the last release we present the new PHPIDS 0.4.6. This time we did lots of optimizations on the generic attack detection and the PHPIDS Centrifuge. There is a pretty new way to detect vectors which are not caught by the rules and as far as we heard we returned some of the headaches our testers gave to us before - thanks again to David Lindsay, Gareth Heyes and Johannes Dahse for their great work.

The rules were optimized as usual and again - they haven’t grown but become even smaller for better performance. Altogether the rules decreased their size by 937 bytes. The converter was optimized too and many smaller bugs were fixed.

You can find the fresh packages here as usual. Again - no API changes so updating should work like a charm.

Also we have continued working on our sister project - the CSRFx. Now this system is even more capable in dealing with invalid markup to protect and AJAX requests. Also JSON wrapped markup can now be secured with the token cloud of the CSRFx so maybe you like to check out the sources and give it try.

We appreciate your feedback and if you happen to have any problems during installation or usage feel free to ask us in our forum.

After the pretty successful Christmas release we now present PHPIDS 0.4.5. It brings a lot of enhancements in vector detection. We worked over the rules and especially the converter and due to the great help of David, Gareth, Johannes and tx many bugs were found and fixed. The exploits and filter circumventions they found were awesome as usual and got our team surprised a lot. JavaScript is a hell of a language - and so is SQL…

We also did some improvements to the PHPIDS Centrifuge. We now have - supporting the main Centrifuge core - an additional layer to detect attacks based on character ratio. Take a peek at the code if you wish to know more details.

The API hasn’t changed in this release so patching would be definitely no problem as usual. We hope you like the new release and grab you package here! Also we heard some bird twitter about a new WPIDS release some when this week - stay tuned!

Just in time before the holidays we are proud to release PHPIDS 0.4.4. After several weeks of testing by the group we populated the new release with small but important features and optimizations. We added support for detection and translation of JavaScript Unicode - undetected vectors like \0061ert(1) Gareth Heyes discovered now belong to the past. Also we optimized the rules to catch the latest concatenation and code injection vectors crafted by thornmaker and tx.

The centrifuge was optimized a little bit more and here and there we heard about vectors that were exclusively detected by this mechanism - so yes, the concept works. Furthermore we discovered several minor bottle necks when dealing with very large incoming strings which of course were removed too for better scalability and performance.

We also increased the code quality - the PHPIDS is now completely coded in PEAR valid PHP constantly monitored by the PEAR package PHP_CodeSniffer. The test coverage is higher than 95% and we also tweaked the generated documentation for better understanding.

We hope as usual that you like the new release as much as we do and wish you a very happy and relaxing holiday. See you next year!

Today we proudly release PHPIDS 0.4.3. This time we invested all spare time we had the last weeks on enhancing the converter and the rules. That means way better intrusion detection and even fewer false alerts then with the last release.

Thanks to the great help from Johannes Dahse we managed to tweak the rules to catch way more SQL Injections - especially the super short ones for authentication bypass and information disclosure. And - we didn’t believe it ourselves - SirDarckCat and Gareth Heyes even found some new XSS vectors slipping through the rules. We also optimized the converter against several evil Unicode characters and other possibilities to obfuscate payload.

Furthermore we did some more testing and optimization on the PHPIDS centrifuge. After several weeks of high traffic beta testing we agreed to remove the ‘beta’-label from this module too. Be sure to grab the files from here as soon as possible

The coming releases will head straight towards 0.5 - the usability and scalability release. We hope you are looking forward for this one as much as we do - and enjoy PHPIDS 0.4.3.

Today we are talking to Reiners who helped us enhancing the SQL Injection detection rules. Thanks to his outstanding work we were able to identify lots of bugs in the rules and make the PHPIDS a lot better in SQL Injection detection that we ever thought it could be.

Q: Please tell us a little bit about yourself?

My name is Johannes Dahse and I am studying “IT-Security” at the Ruhr University Bochum in Germany. Beside my studies I read a lot about websecurity and experiment with it or I write some codes for smaller projects. I also like to work out, and hang out with friends and grab some beers.

Q: During the last weeks we happened to learn to know you as a top notch SQL Injection expert - how come?

It started with learning PHP and MySQL about 4 years ago. Back then, I was already interested in security in general and did a lot of research. While participating at the last CIPHER (a Capture The Flag-style wargame) I noticed that my SQLi knowledge was a bit rusty and started to do more research on it which leads me to PHP-IDS. I learned a lot during the challenge to trick the filters and had a lot of fun.

Q: XSS vs. SQLI can you compare them? If yes, whose impact is bigger?

That is an interesting question. Generally I would say SQLi is more dangerous because it is a server-side problem and can lead to a full takeover. But it depends on what the attacker wants to do, the DBMS and its settings of course. And there may be a lot of scenarios where XSS as client-side attack is way more effective to reach your goals.
You shouldn’t have one of those holes in your webapp anyway, but I’d rather like to know a XSS hole in my app than a SQLi

Q: WebAppSec in five years - any prognoses?

I think WebAppSec is getting more and more important. The amount of web-applications is growing, however, most of their developers tend to ignore web security. Additionally, many people release their personal information at the internet and therefore, the security for these personal data will play a major role.

Q: Whom would you like to invite for dinner and why

Haha, I can think of a couple of lovely ladies I’d like to meet but I guess you are asking towards webappsec. Well, since I really enjoy reading Ronalds blog (www.0×000000.com) I think it would be really interesting having a chat with him. He has some really interesting posts I’d like to talk more about.

Q: Thanks for the interview!

After a pretty long time without releases we finally present the PHPIDS 0.4.2 which ships a long awaited and heavily demanded feature - absolute paths in the Config.ini. Besides this major change we have done tons of improvements to the rules - especially to the rules to detect SQL Injection patterns.

Furthermore we heavily reduced false alerts again - PHPIDS 0.4.2 is the first release that comes without any false alerts from the false alert DB which you guys kindly helped filling with various input and suggestions. Also the PHPIDS is now capable of detection XXE attacks and basic LDAP injections. We also tweaked the converter and the almighty PHPIDS Centrifuge to ease the preparation of possible attack vectors and make the detection process even faster than before.

Our test suite has meanwhile grown to 75 test cases and covers almost any recent detection bypass to make sure no older attacks slip through due to rule changes.

The next releases will aim to ease implementation and usability of the PHPIDS and most importantly to make it more scalable on very large environments - any suggestions or feature proposals are very welcome as usual.

We hope you like the fresh release and like to thank all the people helping with testing and enhancing the PHPIDS - have fun!

Today we are talking to Giorgio Maone who helped us several times improving the PHPIDS filters and converter with elegant XSS vectors. He was the first to break the filter after the XSS contest begun and some weeks before he generated real headaches on our sides by exploiting the name-trick in multiple ways. Ah - and not to forget - he has a cool blog too

Q: Please tell us a little bit about yourself!

A: I’m a senior software developer and CTO at InformAction, an Italy-based IT consulting firm I co-founded in 1998. I love slow food, martial arts, elegant code, listening good jazz and playing bad jazz. Some competitors keep accusing me of being the evil mind behind NoScript, but I plead not guilty: it’s just a setup to scare way Web 2.0+ investors from my company. Ajax, Comet, Widgets and Mashups FTW!

Q: You have a pretty impressive vita as developer and security expert - how did it all start?

A: My dad took home a Commodore 64 almost 25 years ago, and I immediately started hacking it passionately in a few weeks. 50KB RAM and a supernaturally fast 1MHz CPU can do wonders at carving your notions of “code bloat” and “optimization” in stone. My Web debut has been with Mosaic, an Amiga porting of the venerable NCSA browser with no JavaScript interpreter… did you say imprinting?! When I had my first RDBMS experience, injections were almost impossible: SQL statements were embedded in Cobol programs through a pre-compiler and parameters were safely bound to variables. I’d dare say we saw an involution, through Visual Basic to Web scripting, and only recently the
mainstream (the PHP/MySQL crowd) is red discovering prepared statements and parameter binding, which have always been there, e.g. in JDBC.

Q: Firefox 3 will bring us… please complete the sentence

A: It depends on who’s “us”.

• Users: brand new bookmarking, tagging and rating system (Places), malware blocking (Google-powered blacklist, no less!), disconnected web applications, cross-session resumable downloads, UI to disable plugins
• Chrome developers: simplified JavaScript API (FUEL), site-specific preferences, reliable SQL storage (it’s already here, but now we can stop worrying about legacy compatibility)
• Content developers: offline persistence, better SVG, CSS3 and ACID test compatibility, many HTML5 features
• Black hat folks: offline persistence, better SVG, CSS3 and ACID test compatibility, many HTML5 features
• White hat folks: lots of fun!

Q: What would you forbid if you had the chance

A:

• Stupidity
• Corporate greed
• Globalized exploitation (and no, I don’t mean a PHP attack involving superglobals)

Q: The PHPIDS is… please complete the sentence.

A: The Sudoku killer! The new Rubik cube!! Hours and hours of unlimited fun!!! No, really, I believe it’s a very useful project and a great first line defense against generalized attacks, e.g. those targeting popular CMS packages like Joomla or Wordpress whose quality is not entirely under your control. Blacklist-based filters cannot replace good coding practices, and a really motivated and skilled enough attacker can always find a way around sooner or later, but if your sysadmin is attentive and your developers know their stuff, PHPIDS is surely a precious tool to detect suspicious activity and harden your walls before it’s too late.

Q: Thanks a lot for the interview, Giorgio!