PHPIDS » Web Application Security 2.0

Hinnerk Altenburg of epublica has officially released a Perl port of PHPIDS.

It has been released as CGI::IDS Perl module ‘PerlIDS’ on CPAN.org under the OpenSource Lesser GNU Public License (LGPL).

PerlIDS is compatible to the original XML filter set of PHPIDS. During the development they have made some speed improvements to PerlIDS and PHPIDS for the use on really large websites. Their experience of running it on websites with much user traffic could help to improve our converters to reduce the rate of false alarms.

For heavily reducing the server load they introduced a whitelist mechanism to tell PerlIDS which request parameters don’t have to be checked with the expensive regular expressions if they match the whitelist rules.

They’d love to receive your feedback on the Perl port!

It’s been a while – two months to be precise – since we published the last release of the PHPIDS. But the time waiting was worth it – PHPIDS 0.5.3 brings a lot of features – most of them requested by our users.

Besides a numerous minor fixes this release ships support for the SQL Hex-Encodings like 0x426F6F21 – SQL Injection vectors utilizing this kind of obfuscation thus can now be detected and translated without any problems. The PHPIDS 0.5.3 also delivers JSON support – meaning you can flag certain fields as JSON in the Config.ini to make sure the are decoded properly before hitting the rules and neither generate false alerts nor smuggle payload nested in JSON properties. We were able to fix a hell lot of false alerts – mainly by the help of the guys from epublica, our fellow forum users and several other contributors. You won’t imagine how much trouble we had with smilies and other emoticons…

We also optimized the Centrifuge slightly and tweaked the nested base64 detection and translation – so again less false alerts and more impact when real attacks strike.

Max Romanovsky – another forum user reported a problem with AJAX requests and line breaks – and even submitted a valid fix which we of course included too. Gareth Heyes and David Lindsay found a handful of new XSS injections – and Johannes Dahse reported several SQL Injection vectors that bypassed the rules. Thanks for your great support! We also managed to make the rule files a little bit smaller again – just 3 bytes but we guess that’s better than nothing

So – we hope as usual you have fun with this release. Don’t forget to give us some feedback on how the system works for you to help us making 0.5.4 even a little bit better.

Again we are very proud to announce: PHPIDS 0.5.2 is officially out after a lot of changes and improvements on the recent version. Most mentionable is a performance tweak discovered by Ingo Bax that might save you over 60% of computing time in certain scenarios – just by having removed the case-insensitivity regex modifier in the detection process and having optimized the rules for this change.

Also we fixed a lot of false alerts – especially when dealing with frameworks that tend to accept serialized arrays and objects as parameters. Xajax is one of those and you should be able to combine the PHPIDS and Xajax without any trouble anymore. Of course those weren’t the only false alerts we fixed – the rules received some major slenderizing. Also Nick Benson from sla.ckers.org helped us to optimize several regular expressions in the rules – especially among the SQL Injection detection rules.

What makes us most happy with this release is the fact that we didn’t have any false negatives during the last weeks – not a single one. So it kind of seems that the project has reached a state that even we considered to be almost impossible.

There are several interesting ports growing – like already mentioned in the last release post and meanwhile we are in good dialog with the ModSecurity team which will definitely help to improve both tools.

So – we wish you a lot of fun with the new release and look forward for your feedback.

This post is just meant to inform you that there is an article on PHPIDS printed in the most recent issue of our German PHP Magazine.

Its content is pretty much oriented on the white paper we published earlier so it won’t tell you anything new unless you haven’t known PHPIDS before and just want to get started with it. For that purpose, this article should be a perfect guideline as it covers all the aspects that are necessary to install the system on top of an existing application and then work with it in terms of result analysis.

Unfortunately it was written quite some time ago and published just now, so it doesn’t cover all the cool new features that are available since PHPIDS version >= 0.5. That means you won’t find anything on allowed HTML code in user input, which PHPIDS is capable of to detect and differ from malicious script fragments since the 0.5 branch. It’s pretty easy to work with this feature though and you can catch up on it on our website. If you have any and problems or suggestions, you’re more than welcome to address them on the forums.

Finally the next release of the PHPIDS has arrived – meanwhile at 0.5.1.

We fixed a lot of minor bugs and added a whole bunch of new conversion features for more or less esoteric attack vectors. The very interesting issues Gareth Heyes found some days ago are no longer a danger for PHPIDS users – as well as the pretty ugly XSS DoS attempts possible in Firefox 3. Also the WYSIWYG attack detection has been improved and should provide way more reliability combined with less false alerts.

The filter rules now have IDs – which you can of course access with a getter in the filter object. Thanks to the collaboration with epublica the filter rules have now even better compatibility with Perl regular expressions and other dialects.

Besides the addition of the ID-getter we had no API changes – so an upgrade shouldn’t be a problem at all. We hope you like the new release and provide us with tons of feedback as usual. Stay tuned – the next weeks will be pretty packed with news about collaborations with other security solutions.

After several weeks without releases, only smaller rule upgrades and converter patches we finally present the most recent version of the PHPIDS. Most of you would have expected the 0.4.8 – but we are throwing out 0.5 today – why is that?

Easy explanation: we’ve added a feature that has been requested very often and closes one huge gap in the protection layer the PHPIDS provides. We are talking about user input where valid HTML is allowed – even wanted. Like with WYSIWYG editors and other rich text forms. Until now the PHPIDS wasn’t able to deal with this kind of input – too many false alerts were generated and generally we recommended to add form fields with allowed HTML to the exclusions. Not good.

Those times are over – the PHPIDS 0.5 uses the HTMLPurifier to compare the original user input with the purified one to determine the differences and analyze them with the rules and the centrifuge. You can of course chose freely which fields you want to monitor the traditional way and which are allowed to contain valid HTML – just have a look at the packaged Config.ini to see how it works.


…
scan_keys = false
 
; define which fields contain html and need preparation before
; hitting the PHPIDS rules (new in PHPIDS 0.5)
html[] = __wysiwyg
 
; define which fields shouldn't be monitored
exceptions[] = __utmz
…


We tested this feature for a pretty long time – but of course not as long as the way riper components like the rules and the PHPIDS Centrifuge. So – there might be some false alerts and other minor problems to wipe out in the next releases. Please help us improving the system by submitting problems and contacting us about them via mail, forum or group.

Some other mentionable enhancements are optimizations of the Centrifuge, a lot of important fixes of the rules, optimizations of the converter, extended tests for even more reliability and several performance tweaks. Thanks to Hinnerk Altenburg from epublica the rule set is now even compatible with Perl and Python – so there are no barriers anymore for writing ports for several other languages.

So don’t hesitate too long and grab the latest packages from the downloads section. We hope you like this release as much as we do and have great fun and use detecting attacks and reacting on them however you feel like. Big thanks go to Gareth Heyes, David Lindsay and several others for their help on testing the PHPIDS and again finding exotic but working rule circumventions. Also many thanks to all the guys from OWASP Europe and ph-neutral for their excellent feedback and great discussions about the PHPIDS.

Again – no need for a lot of chit chat – here are the PHPIDS / Generic Attack Detection slides from the ph-neutral 0x7d8 in Berlin. Both the OWASP and the ph-neutral were absolutely great conferences. talking to the visitors and speakers gave us a lot of new ideas for coming features and improvements – so stay tuned.

| View | Upload your own

No need for a lot words – here are the slides of the OWASP AppSec Europe 2008 talk about the PHPIDS and its generic attack detection methods. Have fun watching and feel free to post questions and comments. We’ll upload a more detailed description of the so far great conference and the coming ph-neutral 0x7d8 event in some days.

| View | Upload your own

This weekend I got feedback from the OWASP Crew from Belgium. The talk evolving around the PHPIDS Whitepaper was accepted and found a slot in the time line of the OWASP AppSec Europe 2008.

Don’t miss this event if you want to meet team members of the PHPIDS in persona as well as outstanding security experts like pdp, Ivan Ristic, Martin Johns and many others.

The talk will range from 14:40 to 15:20 in the second track at 21st of May 2008. Main topics are the PHPIDS, how it works, what the major benefits and possible drawbacks are and of course how the black-majickish Centrifuge works and how other tools can utilize its logic. We will publish the presentation and if available a video of the talk for all who unfortunately can’t participate.

We are glad to announce the freshest release of the PHPIDS. As you might have expected we did a lot of work optimizing the converter and the centrifuge again. Also the rules were improved slightly to catch several sophisticated SQL Injection vectors Johannes Dahse submitted. Again we have to thank David Lindsay, Gareth Heyes and others for their great work. The system wouldn’t even be as half as good without their contributions and intense testing.

The PHPIDS now performs way better when dealing with UTF7 XSS and especially data URIs with mixed encoding. Gareth and his outstanding Hackvertor managed to create some weird but sophisticated examples of how data URIs can be obfuscated to the max. Don’t forget to check out his amazing tool.

The PHPIDS now also ‘speaks’ Base64 – so no vector obfuscation with this encoding anymore, bad guys! The count of false alerts has decreased amazingly with the new rules so if an incoming string was detected as suspicious by the PHPIDS you can almost be 99% sure that it was an intrusion attempt.

We’d also like to thank the community from our forum for the help on optimizing the system and adding improvements here and there. Be sure to grab the latest packages here – again no API changes by the way so patching will work without any problems.