PHPIDS » Web Application Security 2.0

It’s been a while since we released the latest version of the PHPIDS. We had tons of microscopic changes in the rules and the Converter during the last weeks and months so we decided to wait a little bit to have a diff large enough legitimizing the jump from 0.5.4 to 0.6.

A lot of new formats are being not supported for de-obfuscation – including way better entity handling, more MSSQL obfuscation techniques, JavaScript backslash line breaks and a lot of other nasty things. We also optimized and fine tunes the Centrifuge to provide better results in generic attack detection.

We optimized the rules against a ton of new SQL Injection attack patterns – mostly submitted by Reiners and Roberto Salgado. Although Gareth Heyes and David Lindsay found new and very interesting ways of executing JavaScript and at the same time bypassing the PHPIDS rules – here’s some of these vectors:

this[('eva')+this.status +'l'](/xx.x.x/+name)

1' and 0x0 != mid(user(),1,1) or null/ 'null

<isindex/type=image
xyz=<iframe/src=javascript&#x3a&#x61lert&#x28&#x31&#x29>
onerror=undefined,/\//,outerHTML=xyz src=1>

Furthermore we had a lot of minor changes making sure fewer false positives are being produced. A lot of small bugs were fixed – thanks to our forum users reports and several tickets. Also Christian wrote a great article for the German print magazine c’t about the PHPIDS. A slightly abridges version can be found here.

You can grab the latest copy in the downloads section as usual. Have fun with the PHPIDS 0.6 and feel free to give us feedback and tel us what you think. And last but not least.. thanks a lot to all who helped with this and former releases!

Arshan Dabirsiaghi is a guy who is almost always wrong. Almost. Today he was right the first time for a pretty long time. He was right when stating that I (.mario – not christ1an or Lars) would hate him for dragging me into this 7 things meme thing. Anyway – here’s what’s asked for

We all know the rules for this game – add a back-link for the one who forced you into doing this, state seven more or less personal things about oneself and what’s making this meme so ultimately dangerous: lure seven other people into it. So – let’s just start to get this over with.

• I collected stamps as child. And coins. And phone-cards. Seriously. I started when being about five years old and quit like three or four years later. I have no comparable hobbies today.
• I consider Apocalypse Now! the best movie I have ever seen – and I saw it more than five times. Directly followed by Fight Club – both masterpieces about human behavior in several interesting situations, the past, the future and things that will never change no matter how hard we try.
• The basic idea for the PHPIDS approached when being on a conference and trying to hack a dating site – which surprisingly detected my dirty attempts and warned me. I was that impressed that I wanted to have such a feature for my own sites – the PHPIDS was born.
• I love Aqua Teen Hunger Force. I know – it’s damn 2002 and pretty childish – but the Freudian styled characters – especially Meatwad – get me any time I watch it again. I also dig the other stuff from Williams Street. They really pushed the limit and made me enjoy TV again – at least the American broadcastings.
• I still have nightmares then and when taking me back to the time at the university. Mostly I am happy in these dreams that all the work has been done – and then suddenly I realize that I missed one important exam. Usually I am pretty glad when waking up and slowly fade back to reality. I should visit someone for professional help. Maybe Arshan.
• I love the Big King XXL. That burger-monster from… you know it. I feel guilty while eating it. I feel bad after eating it. I have nightmares the night after (mostly about missed exams… Arshan!) – but I nevertheless like it and eat once or twice a month. Irritating? Yes.
• Right know I am damn happy about the most important decision I made last year. Quitting my job. Finally being able again to create things that either make sense – and not only circumvent existing structures and living patterns for their own sake feels great – and shows me what I missed and learned at the same time.

What – that’s it. Now for the hardest part of this 7 thing task. Bringing misery over seven other people. Here they are:

• fukami – because all the other stuff on your blog is plain boring
• Reiners – if I were a database I would just run when seeing you around. Just run.
• christ1an – I am just curious what you will write.
• kuza55 – the youngest troll I know.
• Click.. – erm – RSnake – for the whole RSnaking.. erm Clickjacking buzz.
• teemow – for being an old man in very tight and short jeans.
• Tim – for making you blog about something else than food recipes.

We thought about adding a Christmas reference to the headline – something like the “Rupert Release” or even worse but thankfully we stopped our selves just in time. So – without further jibber-jabber – here it is – PHPIDS 0.5.4. This is the last minor release before 0.6. The release features a bug fix against a buffer overflow problem with mb_string functions Gareth Heyes informed us about just this morning. Also the HTMLPurifier has been upgraded to its most current release version.

Furthermore we had a ton of smaller bug fixes and performance improvements. David Lindsay and Gareth Heyes also found some more really esoteric circumventions of the XSS detection – that we fixed of course. Some bugs in the logging and caching system have been wiped out – thanks to the help of our forum users and Hinnerk Altenburg from epublica who detected several smaller performance bottlenecks. So you will benefit from better performance, robustness, more logging options, finally working database caching and a lot more details.

We wish you a lot of fun with new release – which you can find in the download area as usual – and hope for your feedback. Also don’t hesitate to send in feature requests for the coming 0.6. We also wish you a great new year to come and happy holidays!

Not much more to say. German readers – feel free to grab a copy to make yourself happy and us incredibly rich

About 620 pages of content pennied by parts of the PHPIDS team and other experts from the web security sector combining more than one year of hard work, sweat, soul and tears. Click the image link to learn more.

Since we have some more free time now you can expect a new relase of the PHPIDS during the next days too. We are getting closer and closer to 0.6 so stay tuned for more news.

Hinnerk Altenburg of epublica has officially released a Perl port of PHPIDS.

It has been released as CGI::IDS Perl module ‘PerlIDS’ on CPAN.org under the OpenSource Lesser GNU Public License (LGPL).

PerlIDS is compatible to the original XML filter set of PHPIDS. During the development they have made some speed improvements to PerlIDS and PHPIDS for the use on really large websites. Their experience of running it on websites with much user traffic could help to improve our converters to reduce the rate of false alarms.

For heavily reducing the server load they introduced a whitelist mechanism to tell PerlIDS which request parameters don’t have to be checked with the expensive regular expressions if they match the whitelist rules.

They’d love to receive your feedback on the Perl port!

It’s been a while – two months to be precise – since we published the last release of the PHPIDS. But the time waiting was worth it – PHPIDS 0.5.3 brings a lot of features – most of them requested by our users.

Besides a numerous minor fixes this release ships support for the SQL Hex-Encodings like 0×426F6F21 – SQL Injection vectors utilizing this kind of obfuscation thus can now be detected and translated without any problems. The PHPIDS 0.5.3 also delivers JSON support – meaning you can flag certain fields as JSON in the Config.ini to make sure the are decoded properly before hitting the rules and neither generate false alerts nor smuggle payload nested in JSON properties. We were able to fix a hell lot of false alerts – mainly by the help of the guys from epublica, our fellow forum users and several other contributors. You won’t imagine how much trouble we had with smilies and other emoticons…

We also optimized the Centrifuge slightly and tweaked the nested base64 detection and translation – so again less false alerts and more impact when real attacks strike.

Max Romanovsky – another forum user reported a problem with AJAX requests and line breaks – and even submitted a valid fix which we of course included too. Gareth Heyes and David Lindsay found a handful of new XSS injections – and Johannes Dahse reported several SQL Injection vectors that bypassed the rules. Thanks for your great support! We also managed to make the rule files a little bit smaller again – just 3 bytes but we guess that’s better than nothing

So – we hope as usual you have fun with this release. Don’t forget to give us some feedback on how the system works for you to help us making 0.5.4 even a little bit better.

Again we are very proud to announce: PHPIDS 0.5.2 is officially out after a lot of changes and improvements on the recent version. Most mentionable is a performance tweak discovered by Ingo Bax that might save you over 60% of computing time in certain scenarios – just by having removed the case-insensitivity regex modifier in the detection process and having optimized the rules for this change.

Also we fixed a lot of false alerts – especially when dealing with frameworks that tend to accept serialized arrays and objects as parameters. Xajax is one of those and you should be able to combine the PHPIDS and Xajax without any trouble anymore. Of course those weren’t the only false alerts we fixed – the rules received some major slenderizing. Also Nick Benson from sla.ckers.org helped us to optimize several regular expressions in the rules – especially among the SQL Injection detection rules.

What makes us most happy with this release is the fact that we didn’t have any false negatives during the last weeks – not a single one. So it kind of seems that the project has reached a state that even we considered to be almost impossible.

There are several interesting ports growing – like already mentioned in the last release post and meanwhile we are in good dialog with the ModSecurity team which will definitely help to improve both tools.

So – we wish you a lot of fun with the new release and look forward for your feedback.

This post is just meant to inform you that there is an article on PHPIDS printed in the most recent issue of our German PHP Magazine.

Its content is pretty much oriented on the white paper we published earlier so it won’t tell you anything new unless you haven’t known PHPIDS before and just want to get started with it. For that purpose, this article should be a perfect guideline as it covers all the aspects that are necessary to install the system on top of an existing application and then work with it in terms of result analysis.

Unfortunately it was written quite some time ago and published just now, so it doesn’t cover all the cool new features that are available since PHPIDS version >= 0.5. That means you won’t find anything on allowed HTML code in user input, which PHPIDS is capable of to detect and differ from malicious script fragments since the 0.5 branch. It’s pretty easy to work with this feature though and you can catch up on it on our website. If you have any and problems or suggestions, you’re more than welcome to address them on the forums.

Finally the next release of the PHPIDS has arrived – meanwhile at 0.5.1.

We fixed a lot of minor bugs and added a whole bunch of new conversion features for more or less esoteric attack vectors. The very interesting issues Gareth Heyes found some days ago are no longer a danger for PHPIDS users – as well as the pretty ugly XSS DoS attempts possible in Firefox 3. Also the WYSIWYG attack detection has been improved and should provide way more reliability combined with less false alerts.

The filter rules now have IDs – which you can of course access with a getter in the filter object. Thanks to the collaboration with epublica the filter rules have now even better compatibility with Perl regular expressions and other dialects.

Besides the addition of the ID-getter we had no API changes – so an upgrade shouldn’t be a problem at all. We hope you like the new release and provide us with tons of feedback as usual. Stay tuned – the next weeks will be pretty packed with news about collaborations with other security solutions.

After several weeks without releases, only smaller rule upgrades and converter patches we finally present the most recent version of the PHPIDS. Most of you would have expected the 0.4.8 – but we are throwing out 0.5 today – why is that?

Easy explanation: we’ve added a feature that has been requested very often and closes one huge gap in the protection layer the PHPIDS provides. We are talking about user input where valid HTML is allowed – even wanted. Like with WYSIWYG editors and other rich text forms. Until now the PHPIDS wasn’t able to deal with this kind of input – too many false alerts were generated and generally we recommended to add form fields with allowed HTML to the exclusions. Not good.

Those times are over – the PHPIDS 0.5 uses the HTMLPurifier to compare the original user input with the purified one to determine the differences and analyze them with the rules and the centrifuge. You can of course chose freely which fields you want to monitor the traditional way and which are allowed to contain valid HTML – just have a look at the packaged Config.ini to see how it works.


…
scan_keys = false
 
; define which fields contain html and need preparation before
; hitting the PHPIDS rules (new in PHPIDS 0.5)
html[] = __wysiwyg
 
; define which fields shouldn't be monitored
exceptions[] = __utmz
…


We tested this feature for a pretty long time – but of course not as long as the way riper components like the rules and the PHPIDS Centrifuge. So – there might be some false alerts and other minor problems to wipe out in the next releases. Please help us improving the system by submitting problems and contacting us about them via mail, forum or group.

Some other mentionable enhancements are optimizations of the Centrifuge, a lot of important fixes of the rules, optimizations of the converter, extended tests for even more reliability and several performance tweaks. Thanks to Hinnerk Altenburg from epublica the rule set is now even compatible with Perl and Python – so there are no barriers anymore for writing ports for several other languages.

So don’t hesitate too long and grab the latest packages from the downloads section. We hope you like this release as much as we do and have great fun and use detecting attacks and reacting on them however you feel like. Big thanks go to Gareth Heyes, David Lindsay and several others for their help on testing the PHPIDS and again finding exotic but working rule circumventions. Also many thanks to all the guys from OWASP Europe and ph-neutral for their excellent feedback and great discussions about the PHPIDS.