According to NASA, the Phoenix Lander has detected snow falling from clouds 4km above its landing site. Whilst the snow is vapourising before it hits the ground, this is a remarkable observation of Mars’ vestigial atmospheric processes in motion.
This is a handy little snippet that allows you to set rate limits for views in Django.
I’m working on a short story about different perspectives over a long period of time at the moment, as was bouncing around the interwebs for inspiration when I came across this little gem from the 2003 Academy Awards for short films. It didn’t win, but it clearly deserved the nomination.
There’s a higher resolution version (definitely recommended for some of the details in the animation) available too.
Beautiful Soup is a great little Python module that will read just about any HTML page and give you back a structured parsed tree. It’s awesome because you can pass it just about any mangled markup – I’ve never known it to choke on anything. For some web service consumers I’ve had to write over the years Beautiful Soup has saved me many, many hours of slogging through crappy HTML parsing. Great software deserves appreciation.
Whilst browsing my good friend Rachel’s website I happened to notice that her brother Leonard wrote Beautiful Soup. He also wrote RESTful Web Services, which is part of my (recently pruned) dead tree collection, and which I’d heartily recommend to anyone who has to work with REST web services. The Django examples were especially useful!
Recently I’ve been putting some time into writing a database adapter for Django that uses Amazon’s S3 and SimpleDB services as a storage layer, whilst trying to retain as much of Django’s QuerySet functional layer as possible. The general goal is to provide a storage back-end for Django that isn’t dependent on the traditional vertically-scaling database server, but can scale horizontally in the same way as the EC2 computing cloud does. My eventual goal being the ability to deploy Django in the cloud with no external dependencies. Just throw out a Django machine image, deploy your app’s code and config, and you have a scaling solution that takes minutes rather than days or weeks.
It’s a non-trivial exercise that is both stimulating and frustrating in equal measure, and progress has been steady, if not exactly rapid. It’s worth it to me though, as the ability to roll out scaling infrastructure is dramatically hampered by the database layer.
Imagine my delight then to find that Google have launched AppEngine, their own cloud-based web application system. It’s Python without any messy machine-based libraries, uses WSGI so you can use pretty much any Python web app, and with GFS for a distributed file storage and BigTable as a data persistence layer. Google even throws in Django 0.96.1 with instructions on how to use their storage layers by doing away with Django’s own model (more on this later).
There’s a lot of whining about how Google’s solution cripples Python (which is crazy when you look at how trivial it is to refactor code to use Google’s supplied alternatives), and locks you into their solution. I suspect that this is mostly from people who have never even contemplated building an application that needs to really scale, and are therefore still thinking in terms services provided by the underlying OS. That’s a big problem for scaling, because disk, IO, threads, sockets, etc are finite resources that are hardware-bound. Abstracting access to these things is tough. Most scaling solutions these days are about providing multiple hardware instances, but unfortunately that only solves the hardware problem. Building an app that scales transparently over multiple hardware instances is a huge challenge in comparison to procuring more servers.
Google’s approach is to do away with the concept of hardware entirely. That means a change of mindset towards every request being an atomic operation. Persistence occurs (correctly) in your persistence layer and not in transient storage available to an instance of your application. Google have provided extensive Python libraries and API calls to enable applications to take advantage of this, but it seems that a fairly vocal group aren’t interested unless their applications work on AppEngine without any additional effort. Considering the paradigm shift that AppEngine represents (from machine-centric programming to distributed programming) it’s not unreasonable to expect some small effort to be required. Especially when you take into account that AppEngine is currently in a very limited trial phase.
I’m extremely optimistic that Google’s approach will work well for a number of reasons. As an application programmer I spend huge amounts of time working around hardware and platform limitations that I should be spending on core functional areas. If Google can provide a solution that means I never have to worry about specific hardware problems ever again, I doubt I’ll look back.
The BBC has just released Flash streaming for iPlayer’s 7 Day Catch-Up feature. This is just about the best news to come out of the fiasco that is iPlayer, and for those of us unwilling or unable to install the bandwidth-destroying Kontiki client it’s the only way we can get our on-demand BBC programming.
Unfortunately, the BBC only makes the streams available on their site. I checked the iPlayer terms and conditions, and there’s nothing stopping remote embedding, it’s just a technical hurdle.
Fortunately, the internet eats technical hurdles for breakfast.
So, in the half hour before I go out to the pub, I’ve knocked together a little WordPress plugin that will take an iPlayer URI, and embed the streaming video into your page for you.
Streaming full-screen Doctor Who on my website. Excuse me whilst I geek-out a moment, this is the coolest thing I’ve played with all week!
I’ll tidy up the code and release it to the community as soon as I get back from a night of drinking and dancing!