PHPIDS » Web Application Security 2.0

Today we proudly release PHPIDS 0.4.3. This time we invested all spare time we had the last weeks on enhancing the converter and the rules. That means way better intrusion detection and even fewer false alerts then with the last release.

Thanks to the great help from Johannes Dahse we managed to tweak the rules to catch way more SQL Injections - especially the super short ones for authentication bypass and information disclosure. And - we didn’t believe it ourselves - SirDarckCat and Gareth Heyes even found some new XSS vectors slipping through the rules. We also optimized the converter against several evil Unicode characters and other possibilities to obfuscate payload.

Furthermore we did some more testing and optimization on the PHPIDS centrifuge. After several weeks of high traffic beta testing we agreed to remove the ‘beta’-label from this module too. Be sure to grab the files from here as soon as possible

The coming releases will head straight towards 0.5 - the usability and scalability release. We hope you are looking forward for this one as much as we do - and enjoy PHPIDS 0.4.3.

Today we are talking to Reiners who helped us enhancing the SQL Injection detection rules. Thanks to his outstanding work we were able to identify lots of bugs in the rules and make the PHPIDS a lot better in SQL Injection detection that we ever thought it could be.

Q: Please tell us a little bit about yourself?

My name is Johannes Dahse and I am studying “IT-Security” at the Ruhr University Bochum in Germany. Beside my studies I read a lot about websecurity and experiment with it or I write some codes for smaller projects. I also like to work out, and hang out with friends and grab some beers.

Q: During the last weeks we happened to learn to know you as a top notch SQL Injection expert - how come?

It started with learning PHP and MySQL about 4 years ago. Back then, I was already interested in security in general and did a lot of research. While participating at the last CIPHER (a Capture The Flag-style wargame) I noticed that my SQLi knowledge was a bit rusty and started to do more research on it which leads me to PHP-IDS. I learned a lot during the challenge to trick the filters and had a lot of fun.

Q: XSS vs. SQLI can you compare them? If yes, whose impact is bigger?

That is an interesting question. Generally I would say SQLi is more dangerous because it is a server-side problem and can lead to a full takeover. But it depends on what the attacker wants to do, the DBMS and its settings of course. And there may be a lot of scenarios where XSS as client-side attack is way more effective to reach your goals.
You shouldn’t have one of those holes in your webapp anyway, but I’d rather like to know a XSS hole in my app than a SQLi

Q: WebAppSec in five years - any prognoses?

I think WebAppSec is getting more and more important. The amount of web-applications is growing, however, most of their developers tend to ignore web security. Additionally, many people release their personal information at the internet and therefore, the security for these personal data will play a major role.

Q: Whom would you like to invite for dinner and why

Haha, I can think of a couple of lovely ladies I’d like to meet but I guess you are asking towards webappsec. Well, since I really enjoy reading Ronalds blog (www.0×000000.com) I think it would be really interesting having a chat with him. He has some really interesting posts I’d like to talk more about.

Q: Thanks for the interview!

After a pretty long time without releases we finally present the PHPIDS 0.4.2 which ships a long awaited and heavily demanded feature - absolute paths in the Config.ini. Besides this major change we have done tons of improvements to the rules - especially to the rules to detect SQL Injection patterns.

Furthermore we heavily reduced false alerts again - PHPIDS 0.4.2 is the first release that comes without any false alerts from the false alert DB which you guys kindly helped filling with various input and suggestions. Also the PHPIDS is now capable of detection XXE attacks and basic LDAP injections. We also tweaked the converter and the almighty PHPIDS Centrifuge to ease the preparation of possible attack vectors and make the detection process even faster than before.

Our test suite has meanwhile grown to 75 test cases and covers almost any recent detection bypass to make sure no older attacks slip through due to rule changes.

The next releases will aim to ease implementation and usability of the PHPIDS and most importantly to make it more scalable on very large environments - any suggestions or feature proposals are very welcome as usual.

We hope you like the fresh release and like to thank all the people helping with testing and enhancing the PHPIDS - have fun!

Today we are talking to Giorgio Maone who helped us several times improving the PHPIDS filters and converter with elegant XSS vectors. He was the first to break the filter after the XSS contest begun and some weeks before he generated real headaches on our sides by exploiting the name-trick in multiple ways. Ah - and not to forget - he has a cool blog too

Q: Please tell us a little bit about yourself!

A: I’m a senior software developer and CTO at InformAction, an Italy-based IT consulting firm I co-founded in 1998. I love slow food, martial arts, elegant code, listening good jazz and playing bad jazz. Some competitors keep accusing me of being the evil mind behind NoScript, but I plead not guilty: it’s just a setup to scare way Web 2.0+ investors from my company. Ajax, Comet, Widgets and Mashups FTW!

Q: You have a pretty impressive vita as developer and security expert - how did it all start?

A: My dad took home a Commodore 64 almost 25 years ago, and I immediately started hacking it passionately in a few weeks. 50KB RAM and a supernaturally fast 1MHz CPU can do wonders at carving your notions of “code bloat” and “optimization” in stone. My Web debut has been with Mosaic, an Amiga porting of the venerable NCSA browser with no JavaScript interpreter… did you say imprinting?! When I had my first RDBMS experience, injections were almost impossible: SQL statements were embedded in Cobol programs through a pre-compiler and parameters were safely bound to variables. I’d dare say we saw an involution, through Visual Basic to Web scripting, and only recently the
mainstream (the PHP/MySQL crowd) is red discovering prepared statements and parameter binding, which have always been there, e.g. in JDBC.

Q: Firefox 3 will bring us… please complete the sentence

A: It depends on who’s “us”.

• Users: brand new bookmarking, tagging and rating system (Places), malware blocking (Google-powered blacklist, no less!), disconnected web applications, cross-session resumable downloads, UI to disable plugins
• Chrome developers: simplified JavaScript API (FUEL), site-specific preferences, reliable SQL storage (it’s already here, but now we can stop worrying about legacy compatibility)
• Content developers: offline persistence, better SVG, CSS3 and ACID test compatibility, many HTML5 features
• Black hat folks: offline persistence, better SVG, CSS3 and ACID test compatibility, many HTML5 features
• White hat folks: lots of fun!

Q: What would you forbid if you had the chance

A:

• Stupidity
• Corporate greed
• Globalized exploitation (and no, I don’t mean a PHP attack involving superglobals)

Q: The PHPIDS is… please complete the sentence.

A: The Sudoku killer! The new Rubik cube!! Hours and hours of unlimited fun!!! No, really, I believe it’s a very useful project and a great first line defense against generalized attacks, e.g. those targeting popular CMS packages like Joomla or Wordpress whose quality is not entirely under your control. Blacklist-based filters cannot replace good coding practices, and a really motivated and skilled enough attacker can always find a way around sooner or later, but if your sysadmin is attentive and your developers know their stuff, PHPIDS is surely a precious tool to detect suspicious activity and harden your walls before it’s too late.

Q: Thanks a lot for the interview, Giorgio!

It’s not that long ago that we released PHPIDS 0.4 but nevertheless this release brings some very interesting new features you may like. To make things short - here’s a list:

• Getter and setter methods for the config array
• Several completely reworked rules - thanks to the heavy testing in the PHPIDS Group
• Way less false alerts, more true ones with fewer rules
• A pretty new feature called ‘PHPIDS Centrifuge BETA’
• More tests and better coverage
• Simplification of the settings for the Caching features
• Several minor bug fixes

The PHPIDS Centrifuge BETA is basically a new method in the Converter class which is able to detect unknown attack patterns not covered by the rules yet. This is performed via acting on incoming and suspicious string like a centrifuge- stripping all uninteresting characters, normalizing some others and finally having a result that is very short and reflects around 75% to 85% of the attack vectors we tried during testing - and we tried a bunch of. We are still testing and enhancing that feature so the minimum amount of chars for a string to be mangled by the Centrifuge is set to 80. Thanks Martin and Gareth for your help with the testing.

Furthermore many users requested the possibility to have a setter for the config array inside the Init object. This is also included in this release and opens the 0.5 sprint we named ‘Usability & Scalability’.

We hope as always that you like this release and thank you for your support. You will find the download packages here.

This time we talk to a guy who approached in the group a while ago and submitted tremendously obfuscated JavaScript vectors and pointed out important flaws in the recent rule revisions. Since he came up with vectors containing characters like ‘ä’ and ‘ö’ we were pretty sure is from Europe - now we know more. Here’s what xorrer has to say:

Q: Please tell us a little bit about yourself!

A: I’m a software engineering student at the Vienna University of Technology. Currently I’m working for a small company on their web shop. I like to listen to all kinds of music, my all time favorites are The Doors.

Q: How did you come to webappsec

A: To be honest, a few weeks ago I wasn’t at all into webappsec. I guess I was reading/researching some stuff for work when I came across some RSS-feed about XSS. After a little research, a few other feeds (e.g RSnake’s XSS cheat sheet) I was into it. Since then I read a lot about PHP Security, SQLI, XSS, you name it. And with this entry I got to the PHPIDS project.

Q: What do you think about JavaScript - especially at the moment

A: That’s a funny question, as before writing some vectors for PHPIDS I never really did anything in JavaScript. Now after some fiddling around, while testing PHPIDS, I think that JavaScript is the ultimate web security nightmare. As Gareth already stated “Javascript: The ultimate hacking tool“. There is just way too much you can do with it and nowadays you can’t even turn it off in your browser, as there are many sites out there which won’t work without (Ajax, jQuery, Dojo, …).

Q: You’re hat is… please complete the sentence!

A: White. Responsible disclosure is the way to go. There are just too many script kiddies and other people out there for full disclosure.

Q: How would you imagine webappsec in five years?

A: I guess that for some years to come the situation will remain basically the same. The protections are just inferior to the attacks. And this won’t change so easily, as long as no company really gives (or even knows they should give) a thing about web security. Everyday there gets launched a new service and a new technology and in that rate there will be new un-thought of vulnerabilities and new attacks waiting.

Q: The PHPIDS is… please complete the sentence!

A: A good weapon for the first row of defense. It’s build by people who know what they are doing and tested by experts on the field of web application security.

Q: Thanks a lot for the interview!

Today we are talking to thornmaker. He is relatively new to the group and managed to evade the filter rules several times with JavaScript concatenation vectors - which can be described as very sophisticated and clearly near the edge of readability. You can take a look at his work here and here.

Q: Please tell us a little bit about yourself

A: I am David Lindsay (thornmaker). I am happily employed at Security
Innovation, an application security company based out of Boston, Seattle,
and Amsterdam. While in school I studied pure math (modern algebra, number
theory, topology, and so forth). I am married and became a father 1 year ago
which means a lot of my spare time now is devoted to my little pesky one.
My primary interests right now are web app sec, cryptography, mathematics,
genetics, astronomy, and AI.

Q: How did you get into web technologies and JavaScript

A: After finishing school, I worked in QA for a while testing Java Applications
which is where I became interested in web application security. I have not
had any particular affinity to JavaScript until the last couple of months,
largely thanks to the PHPIDS project.

Q: The hat on your head is usually…

A: …displaying a black and white penguin, with a little bit of yellow on the
beak and feet. Single colored hats are so… bland

Q: What’s the biggest current problem in webappsec

A: Not enough companies taking web app sec (or even security in general)
seriously in the first place.

Q: The PHPIDS is a … please complete the sentence

A: …an excellent project and also a good example of why you can’t rely upon
blacklisting to prevent XSS. I actually only started to look at the project
itself a couple of days ago. My primary interest in the project all along
has been in bypassing the filters simply because I find it a difficult and
rewarding challenge. To that end, thank you for your vigilant attention to
keeping the filters updated!

Thanks a lot for the interview!

After several weeks of work, dozens of sleepless nights, discussions and coding sessions we finally present you the brand new version 0.4 of the PHPIDS. We implemented a whole bunch of new and useful features which improve the PHPIDS in several ways.

First of all you will never have to edit the sources of the PHPIDS core files if you want to configure a certain value - because now theres a Config.ini. This file allows you to configure all important settings in a usable and flexible way. The download package ships a version which you most probably can use out of the box - but make sure the file isn’t located inside the web-root on accident.

Furthermore the PHPIDS now provides an advanced caching interface - be it file caching, database caching, session caching use only if you know exactly what you do) or even memcache caching. As well as all other settings you can chose your favorite caching type in the Config.ini. Our benchmarks resulted in performance boosts between 30% and 45% - depending on the caching type.

Of course we did some major improvements to the filter rules and the conversion algorithms - after a nerve grinding contest in the PHPIDS group we are now more than content with the current rule set. Especially when facing the fact that we reduced the number of rules and straightened up the existing ones.

With all the new features we had to change the API slightly - so be aware when upgrading. The example.php and the FAQ should help you with this - altogether just a small handful of lines have to be changed when upgrading from 0.3.x to 0.4. You will also find a complete generated documentation in the docs folder that might help you on this and other issues if necessary.

Download PHPIDS 0.4

The PHPIDS Team would like to thank all contributors - especially the guys who managed to circumvent the rules the last weeks with more than sophisticated vectors and helped us to improve the rues. Same goes for the guys from Zend.com which helped us with advice and field tests.

We hope you like the new release - if any further questions pop up don’t hesitate to contact us!

Today we talk to Kishor Datar. He joined the PHPIDS Team pretty early and since then provided great help on improving the filter rules, writing tools to test the quality of the PHPIDS.

Kishor is well known in the webappsec scene and maintains an interesting blog about security in general. He is furthermore the author of the XSS in eXceSS tool which is a number one reference for developers who want to get in touch with XSS.

Q: Please tell us a little bit about yourself

A: Shorter version of my name is Kishor Datar. I am a Masters student at University of Maryland Baltimore County. Well I just have about two years of experience in software industry. I know (rather knew) Indian Classical Music a little bit. And I try to learn guitar on my own.

Q: how did you get involved in webappsec

A: I had developed a web app firewall as my undergrad project. It was based on regexes like PHPIDS is. But it wasn’t perfect if I may say. In fact I knew nothing about XSS back then. So we had very simple rules that blocked script tags and some more things. Also since we built the proxy ourselves, we didn’t handle most of the other things like encoding etc. Then I worked with a company that built web security scanner for two years. There I learned more about web security. About one year back I started blogging. So I am relatively new to web app sec.

Q: what color is your hat usually and why

A: White hat. Unless the issue is very trivial I don’t like to go for full disclosure.

Q: where do you see webappsec in let’s say 5 years

A: After five years, collaboration between web apps will increase further. They will become more and more flexible. Therefore security will be an important goal in web app development cycle. And therefore web app sec will also maintain its importance.

Q: what do you think of the PHPIDS and related projects

A: Since the time I started blogging and getting involved in web sec community, I have seen people talking about only the problems (IMO). They either talked about how to exploit something or tools that found security holes for you. They rarely talked about the solutions. PHPIDS I feel is different than these projects. Because it actually produced something useful. I personally learned a lot from it. And I enjoyed decoding the complex vectors that others developed. And the time they take to fix a vector is how much? Just 15 mins? Thats really impressive.

Q: Thanks a lot for the interview!

This is the second part of a series of interviews with people from the PHPIDS group. Today we are chatting with Gareth Heyes.

He recently submitted a bunch of concatenation vectors which gave us real headaches and helped a lot to improve the rules and the converter unit. Gareth is pretty well-known in the security scene and created various tools like the CSK and the JSFuzzer

You can get a taste of his work here and here. Have fun reading!

Q: Please tell us a little bit about yourself?

My name is Gareth Heyes and work for a big insurance firm near Manchester. I spend most of my daily working life developing web applications and learning new programming techniques. I’m always hacking my own stuff and trying to improve my code on a daily basis. My security work is done in my spare time and I often spend many hours in front of a monitor into the early hours of the morning. I’m married, have two dogs and I love playing and watching football when I’m not hacking.

Q: Who are The Spanners?

My mate Jake Smith asked if I wanted to be involved in a community web development blog, the name came from the “span” HTML tag, the original goal was to create a blog to enable designers and programmers to share knowledge. I started to post a lot of security stuff because I was really into doing security research and it expanded from there. I feel a bit guilty for taking over the blog but Jake assures me that he doesn’t mind and enjoys reading my stuff.

Q: You in ten years - looking back at WebAppSec today. What would you say?

I’d say it’s much harder to secure a web site than it was 10 years ago, the browser security model needs to improve because the
attacks are always getting better. The basic browser security model hasn’t changed much and that worries me a great deal, even 10 years ago the same attacks will work now. One thing that hasn’t changed much is vendors, I submitted XSS attacks to AOL, MSN and Altavista 10 years ago and never got a reply!, at the time I didn’t know it was XSS, I just found that I could insert HTML into their search
results.

Q: JavaScript is… please complete the sentence!

The ultimate hacking tool

Q: You spent lots of time in creating supreme vectors to test the PHPIDS - why?

Lots of reasons really, I love the project and the fact that it is open source and the challenge of beating your filters after so many talented people have submitted vectors. Most of all though I was tempted by Sirdarckcat’s post, I love being challenged to do something that people think is difficult or impossible and I suppose that is why I like security research so much.

Q: Thanks a lot for the interview, Gareth!