Prediction: 1st Drupal contributer in space before 2020

On this summers Chaos Computer Club Camp, Nick Farr made the prediction "First hacker on the moon by 2034". This was widely publicized and ridiculed and even though I love these hackers and rocket enthusiasts spirit I'm not sure what to make of it... I like predictions and love the tradition of new years predictions on drupal.org. After attending Drupalcon London on the way back in the train the thought hit me. With all these commercial space flight ventures popping up, Drupal's growing exponentially for many years now: When do these trends cross and we can all cheer at the first Drupal community member wealthy enough to go where no Drupalist has gone before?

I know one thing for sure: It won't be me. I'm more a thrill avoider than a seeker, but I do have that spark of enthusiasm for space exploration that many nerds share and I am sure the waiting lists are already filled with people that have made their fortunes in the hight tech and internet business. A few of these commercial space flight are now gearing up their operations and within a few years these should be more a commodity then a novelty. After the initial hype is over prices should drop from billionair to miljonair levels.

Every time I attend a Drupal event, be it in real life or online, I am astounded by the growth in both absolutes and the level of diverse talents that have come to our once small community. Being surrounded by these amazing, intelligent, profesional and fun people it's clear all of them are doing well economically. Even, or some say especially, in economic downtimes Drupal seems to defeats all trends and maintain an incredible growth rate. Drupalists from regions like Spain, Portugal, Ireland, Greece are just as desperate for more good people with Drupal (related) skills to keep up with demand for more Drupal sites. And the big shops are still growing, hour rates are rising, clients and their projects getting bigger, with no end in sight.

I have not done any real computation or statistical analysis here but my gut feeling is these Drupal people will keep doing well and I'm sure it's just a matter of time before a Drupal community member will board that low orbit plane he or she will have paid significant piece of their hard earned money for a seat on it and experience those few minutes of 0 gravity. So here it is: A Drupal community member in space before the end of this decade. I am taking bets, just leave a comment or tweet me.

Purge: Clearing Drupal pages from Proxy Caches

For some time I've been working on a little module I named Purge. Here's a little background on that effort and what's it been like to publish and improve it out in the public on drupal.org.

It's my first published drupal code contribution, it's my first serious programming project in the many years since I'd decided to drop out of a programmer eduction and career in the second half of the '90. Since then I've mastered many skills in system administrating and network architecture. Though a flawed web 2.0 startup project around 2007 I've run into Drupal and I've been using and evangelizing it's use ever since. In my current occupation as system operator for the web hosting department of a big IT firm I've been working with Varnish, an open source and extremely fast reverse proxy cache software package. In an in-company training by Linpro (who have now spun off their Varnish support business as Varnish Software) I learned about "Purging", a command to clear an object from the cache so it will get refreshed on the next request. Varnish allows purges to be issued via the administration interface and through http requests.

This all led me to test the Varnish Drupal contrib module. It integrates with Varnish on the administration socket and allows for purging. In fact it will purge the complete cache for a domain on any content update. It however also allows for more precieze purges of individual URL's when configured to work with the Expire module. I liked the functionality of the module but foresaw some major issues when imagining this for allowing Drupal sites we host at my job to purge from our shared Varnish platform. The use of the administration interface would imply long and difficult discussion with our security department about opening up network ports for it. Having to add to this that the interface has very weak security and no privilege separation between our clients this would probably be a no go.

So I started to hack the option for HTTP purging into the Varnish module. I managed to get it working but during development I found out this purge method can be implemented in Squid and Nginx as well. This and the lack of interest by the Varnish module maintainers made me decide to branch off into a separate module. I dropped the functionality to purge a complete domain and went for exclusive interfacing with the Expire module that provides nice hooks to plug into. I also decided to do all http requests using curl and curl_multi objects to fire requests simultaneously whenever possible to minimize performance impact.

I first had to apply for a CVS account. In mid 2010 Drupal was still using this legacy versioning system and getting access to publish a module had to be gained by a thorough vetting process that took me a few months and a few iterations of code improvements to pass. The process itself required a lot of patience and some lobbying inside my Drupal network but in the end helped me to improve my code style and the module itself.

When finally publishing the code on drupal.org the benefits of open source continued. A few people started using it and some reported back bug reports with fixes. I also received feature requests (also with patches) that helped me broaden the potential use of the module. It now supports Varnish, Squid, Nginx and has support to add headers for the Acquia Cloud hosting services.

When discussing about adding drush and rules integration with Mike Carper, the author of Expire he handed me commit rights on the expire project to avoid code duplication, thus I've written the drush integration for expire. I've also taken it upon myself to restart a drupal handbook section on Reverse Proxy Caches and Varnish in particular so I can refer to it in the modules documentation.

The unwritten road map in my head for Purge includes a port to 7.x, removing the hard dependency on curl and adding some user friendliness to the configuration form. Next to that I'm researching a new web standard draft called LCI (Linked Cache Invalidation) that might be a long term solution to this problem. Will probably end up as a separate project.

Varnish 3.0 Release Party!

Today we publicly unveil the community website for the Varnish 3.0 Release Party. This was a true community effort and to preempt people who will point fingers at me saying I'm the guy who did it: Let me tell you how this came to be.

I'm active in the Drupal community and helped to create some buzz around the Drupal 7 launch earlier this year. For that two people who I happen to know from Dutch Drupal meetups and european Drupalcons, Roy Scholten en Baris Wanschers designed and built the awesome Drupal 7 Release Party website. It was a big succes and over 300 parties were planned all over the world, including one in my hometown, were both Roy and Baris were present too.

At the job we use Varnish Cache, an awesome piece open source software that makes many of our customers sites very fast and efficient. Then just under 2 weeks ago I spotted this message by Rubén Romero on the Varnish mailinglist asking people to think about organizing a party, somewhere mid juni to celebrate the release of the next milestone 3.0 release of the Varnish Cache software. The idea struck me immediately.

I know the Drupal and Varnish community overlap not just within me. The presentation of Varnish architect and main developer Poul-Henning Kamp at Drupalcon Copenhagen last year was well attended and received and Drupal is used on the Varnish website and Varnish Software AB and Acquia, companies that provide support for Varnish Cache and Drupal respectively have a strategic partnership. This should be doable I thought. And it was... Over the next few days I passed along some emails to Roy, Baris and Rubén. Baris provided me with a stripped copy of the original site and over the course of last weekend I re sculpted the original to fit to the needs of the Varnish 3.0 party people. Earlier this week we got the pledge to host the site and Rubén mobilsed the hard working people on the Varnish side to get the DNS settings right and provide us with great artwork.

The site is built on top of PressFlow, a version of Drupal 6 that is better suited to work with reverse proxy servers like Varnish. It uses OpenLayers to display all the events on geographical maps, has OpenID and Twitter integration and should soon have Flickr functionality. It's hosted on a dedicated VMWare virtual machine running RedHat Enterprise Linux with Apache, MySql, and PHP with APC. And off course it's served through Varnish Cache. A fully load balanced, redundant Varnish Cache platform with a lot of bandwidth to spare.

The result is a fully functional website in just a few days of volunteer work. This would however not have been possible without the work of the Drupal community who built the awesome software that made this site, the incredible work of Ronald, Roy and Baris who built the maternal site, the energy of Rubén and the awesomeness of Varnish 3.0 that inspired me to do this and good things it will bring to the internet and my day to day job.

Gearing up for DrupalJam 2011

Mark in your Schedule: June 20th 2011 will be the day of DrupalJam 2011. This year it will be hosted the "Instituut voor Beeld en Geluid" in Hilversum. I'm member of the organizing team and will be responsible for the technical side of things. We plan to bring in people from the Network Event Kit so we'll have awesome wifi for attendees and fixed networking for speakers.

Welcome to the new legacy free SqyD.net

Yes, you're here and you can read this blog post. That is all functionality for now... Nothing exiting so far. The interesting stuff is under the hood. It's running on Drupal 7 on a fresh new host that runs a varnish cache instance in front of it. It's also fully ipv6 ready. Just needs some content...

Subscribe to Front page feed

Support Drupal

I support Drupal!

Drupalcon Munich

I am going to DrupalCon Munich!