They never work as intended the first four times you run them.

My girlfriend runs a blog on a cheap hosting firm that doesn't provide any way of doing proper SQL dumps of the MySQL database used by the blogging software.

There are plugins for Wordpress that can do full backups, but I prefer doing raw SQL dumps + a filesystem backup. That way, you know what you get, you don't have to trust the backup plugin author to do it right.

The hosting firm does provide access to a phpMyAdmin installation which you can use to download SQL dumps. The trick is of course to do this automatically, as good backups need to be unattended.

I wrote a python program that can do this, using what turned out to be an excellent library for programmatic web browsing: mechanize.

The backup script is available in my misctools project on GitHub.

This website runs on a virtual machine I buy from Slicehost. I've also choosen to use their DNS servers for my domain - the service is stable and included in the price.

The Slicehost DNS can be modified using the Slicehost API. I wrote two small scripts for easy modification of Slicehost DNS entries from the commandline or from scripts.

• update_entry, for adding or updating existing entries.
• dhclient_update_hook, which very easily can be used to update an entry from a dhclient script, to keep records that point to dynamic adressess updated automatically.

Both are available from by cloning my misctools project at GitHub.

When trying to get my local development copy of this website running after upgrading my Ubuntu, I got the following confusing error message from the psycopg2 python module:

psycopg2.OperationalError: could not connect to server: No such file or directory
        Is the server running locally and accepting
        connections on Unix domain socket  "/var/run/postgresql/.s.PGSQL.5432"?

My django settings file was correct:

DATABASE_ENGINE = 'postgresql_psycopg2'  # 'postgresql_psycopg2', 'postgresql', 'mysql', 'sqlite3' or 'oracle'.
DATABASE_NAME = 'dbname'                 # Or path to database file if using sqlite3.
DATABASE_USER = 'dbuser'                 # Not used with sqlite3.
DATABASE_PASSWORD = 'dbpassword'         # Not used with sqlite3.
DATABASE_HOST = ''                       # Set to empty string for localhost. Not used with sqlite3.
DATABASE_PORT = ''                       # Set to empty string for default. Not used with sqlite3.

Confusing, since my Postgres server was running and I could connect using psql:

psql -U dbuser -W dbname

This turned out to be one of these problems when Google is of no help - others had the same problem, but I could only find posts where people asked the question, no posts where the actual solution was found.

The cause of the problem was that my PostgreSQL installation was configured to listen on port 5433 instead of the default 5432, and as seen in the error message, the port number is part of the path to the unix socket. The different port was probably setup when I upgraded my Ubuntu, since that installed PostgreSQL 8.4 without completely removing PostgreSQL 8.3. The latter is configured to listen on the default port.

The solution is to either configure the running PostgreSQL to listen on port 5432 by modifying /etc/postgresql/8.4/main/postgresql.conf, or by modifying the Django configuration by setting the port:

DATABASE_PORT = '5433'

For a while, I used Duplicity to make backups to an Amazon S3 bucket. That kind of worked, but I had to do a lot of scripting myself to get it working automatically, so after finding out about Jungledisk, I switched to that. Jungledisk has a nice little desktop applet that keeps track of doing my backups while my computer is on, etc. That's convenient.

Anyway, the Duplicity/S3 experiments left me with an Amazon S3 bucket with about 9000 objects. Getting rid of that proved to be something of a challenge - you have to delete all objects inside the bucket before you can delete the bucket itself, and there's no API call for doing that. I also tried the web application for managing buckets, S3FM but that didn't cope too well with that many objects - my web browser just hung.

I have to admit I could have put more effort into googling before solving it by writing my own script - but writing my own script was more fun :-).

My script managed to delete all 9000 objects without trouble, although it did take quite a while to complete - I let it run overnight.

If you need to do the same thing, it's available here:

http://github.com/forsberg/misctools/tree/0d6cbd189b80e52501d94742fa04bba36b7f34a6/S3

StackOverflow has several other solutions: http://stackoverflow.com/questions/27267/delete-amazon-s3-buckets

I got a new toy today - a GPS logger. Got it as part of a magazine subscription deal - two issues + GPS logger for about $12. That magazine was not my kind of magazine, but getting a GPS logger delivered home for $12 was worth having to throw away two magazine issues :-). For other people trying to get this to work under Linux, the magazine in question was Aktiv Träning.

After some fiddling, I came to the conclusion that the device is based on the SkyTraq Venus chipset:

skytraq: Venus device found: Kernel version = 1.3.3, ODM version = 1.4.5, revision (Y/M/D) = 07/12/11

After plugging the device into an USB port, it appears as a serial port, in my case /dev/ttyUSB0. The data can be retrieved using GPSBabel, but it has to be the development version - the stable version does not feature the skytraq driver required. The development version is available via CVS from sourceforge, see http://sourceforge.net/projects/gpsbabel/develop for details on how to access CVS.

Once you have the development version of gpsbabel, getting the data is as easy as:

./gpsbabel -D 9 -i skytraq,initbaud=38400,baud=38400,erase -f /dev/ttyUSB0 -o gpx -F out.gpx

This will write the GPS coordinate log to out.gpx in the GPS Exchange format - a lightweight XML format.

The above command will also erase the log from the device, so the next download contains just the latest log.

Since I like playing with Google Earth, I also learned that the GPX can be converted into KML - the format Google Earth can read, by running:

./gpsbabel -i gpx -f out.gpx -o kml -F out.kml

Just open out.kml with Google Earth, and you'll see where the GPS logger has been.

On popular request, here's a few images of the device.

The label inside the battery compartment has the following text:

GT-750F/L-Lite
GPS Receiver(Data Logger)
RoHS CE FCC ID:VHP-750F
Made in Taiwan CANMORE

Playing with my GPS logger, I wanted to know the distance travelled between all the points in a GPS log. For example, if I went for a bicycle trip and brought the logger, it would be nice to know how many kilometers I have travelled.

Trying to find a tool that did this without being cumbersome to use proved more difficult than I thought, so I did a quick hack with the help of the python bindings for gpsd. It's a very simple script:

$ ./distance.py promenad2.gpx
427.077378454

Given a GPX file, it will print out the number of meters travelled.

Fetch it in the fgpstools repository on github

If you run a modern computer, chances are high that you have a multi-core processor. That forces us into paralell programming to get more performance, which is much harder than doing serial programming.

The operating system and existing shell commands can do much for us, though. Let's say you have a bunch of files where each should be processed by a command. Of course, with multiple cores, this could be sped up by running multiple instances of the command simultaneously. And there's a very easy way to do that - xargs. An example for converting raw files into jpeg:

find . -type f -name '\*.CR2' \|xargs -n 1 -P 3 ~/bin/raw2jpeg

This will find all files named *.CR2 and feed that list to xargs, which will run up to three (-P 3) processes simultaneously, giving each command one filename (-n 1) on its commandline.

The hotel where I'm staying in Oslo has a very special kind of Wireless Internet. It's the kind where you bring your laptop out in the corridor, connect to the network, and then bring the network with you into your room.

Connecting directly from the room doesn't work, even though there's plenty of signal. Well, at least sometimes, there's plenty of signal. Tonight there is, yesterday night I couldn't see the net.

Must be some norweigan special setup.

The fridge on the picture is probably the most noisy fridge in northern Europe. You'll find it in one of the rooms at Cochs Pensjonat, Oslo. It spends its days, and especially nights, starting with a loud bang about once every 15 minutes, which very effectively wakes people up if they happen to sleep in the room, something people tend to do in hotel rooms.

Oh, how I love to fill the internet with worthless bullshit!

I'm on a bus4you with destination Oslo for a visit to Opera Software's main offices. The bus has wireless internet. That's good, I've been reading webpages on map-reduce, distributed filesystems and other interesting stuff the whole trip.

The bus driver mentioned that the network sometimes stops working, and requires a restart when that happens. It happened, and after confirming with one of my co-passenges that they too had trouble, I went to tell the driver.

No problem, he said, I'll restart it. And to restart it, he pressed a button marked "WWW".

So, if the WWW doesn't work for you, now you know where to find the button with which it can be restarted - just find the nearest bus4you bus! :-)

You talkin' to me? A Blog Challenge? Eh? (Bear, Kolmården Zoo, 2005)

I'm supposed to take the sixth image in the sixth folder in my photo collection, post it here with a comment, and then challenge six other people to do the same thing.

But.. that's an Internet Worm. Naturally, being a responsible computer scientist, I can't help spreading a virus! This blog post is an antivirus.

Still, I'm more than happy to show you, dear reader, some of my pictures. I'll even write a comment for each and every one of them, following the rules of the challenge to some point..

The owner of these boats obviously found a theme. Nessebar Bulgaria. Don't go there, it's a horrible touristish place

Sunshades in the morning. Again, this is from Nessebar, Bulgaria, where you should not go since it's full of tourists and restaurants that all have EVERYTHING as their speciality. But, if you sneak up at 0600 in the morning and spend an hour or so taking photos, you can find some nice angles..

SOF Linköping, 2005 (Student Orchestra Festival).

This horse was having lot's of fun, rolling around on the ground. But now it's decided it's time for some serious.. running around all over the place!

Bah.. getting tired of finding/scaling images. I could keep on for a while, with over 11000 images in my image database..

This will have to do. Not spreading this virus further by challenging any other bloggers, at the same time maintaing a high level of nerdyness!

I went to a seminar about Scrum today at ENEA. It was one of those "let's have a seminar and then give people free food and beer so they buy more consultants from us" type of events.

Even more interesting was the questions after the seminar. Lot's of people from different tech companies in Linköping. Someone said that Scrum kept the programmers happy which would produce better code. That's probably true. Here comes the fun part - another person in the audience were worried that happy programmers would code things they thought were fun instead of the things they were supposed to do.

Hmm.. yeah, right. That's the way it works. Or maybe not! I'd say that the risk is much bigger that bored programmers spend their time working at things they shouldn't do.

I would really like to know where this person works, so I can avoid working there.

The ATI fglrx driver I run against my ATI RS690 chip in my dualhead setup definitely has its quirks. When resuming from disk, both monitors must be turned on when I press the computer's power button, or the X server will behave very odd.

Hard to describe what happens, but when moving the mouse, the whole desktop image is moved instead of only the mouse pointer.

Hardware! I hate hardware! I want all of it to be emulated in software.

For a long time, I was a Plone user and developer. I tried hard to keep myself convinced that Plone was the solution to many web-related problems, not only strictly Content Management-related ones, but all sorts of web applications.

A while ago, I realized that I was, just that - trying hard to keep myself convinced. That is really not a good sign when it comes to frameworks you use in your programming - if you're to work with them and get productive, you have to like them because they actually are good, and fun, to work with. This is not the case with Plone, and I'll try to explain why.

After starting to write this, I realized that I have so many reasons to leave Plone that this will have to be the first in a series of posts. Stay tuned!

The Learning Curve

I started using Plone while looking for a good solution for the external website of the company I was then working for. After some evaluation of different alternatives, I decided on Plone. Plone was then at version 2.0.6, if I recall correctly.

We hired a summer intern with some previous Plone experience with the task of doing different kinds of integration with our existing systems, including some LDAP integration. I remember that he was spending lots of time cursing especially the LDAP integration and the user management code.

About the time when the integration were finished and we began thinking about deplying, Plone 2.1 was released.

KABOOM - an explosion of new things to learn!

Plone 2.1 was like an entirely new piece of software. Lot's and lot's and lot's of things had changed. At least that was the impression you got if you were not a Plone Guru.

I made the decision of going for Plone 2.1, since I didn't want us to be left behind with incompatible Products (that's the Plone term for more or less useful addons that can be downloaded and installed in a Plone instance - I'll probably write something about them later..). So, lot's of time had to be spent again trying to fixup the site into a usable state. The LDAP integration we hoped for had to be half-ditched because of time constraints, and we also had to lower the ambitions on the creation of a nice-looking graphic appearance, instead reusing some old CSS we alread had. Even then, fighting the Plone styles system was a pain, and it didn't really get better in later versions.

We finally got the site deployed. It didn't get updated as often as I had hoped for, and frankly, it didn't work that well, with hickups and performance trouble.

After a few years, Plone 2.5 arrived.

KABOOM - an explosion of new things to learn!

This was where the magical mystery tour of Zope3 technology began to show its ugly face in the code. You had to learn stuff like adapters, interfaces, etc. And although these are well known concepts in the programming world, the key is to neither overuse them nor overcomplicate!

I don't remember, but I think I managed to lift the site up to Plone 2.5. I was really starting to get tired of Plone at this point.

The fun however didn't end there, because some time later, Plone 3 arrived. And again.. KABOOM - lot's of new things to learn. To be just, some of the new things were already familiar if you had learnt them in Plone 2.5, but there were still plenty of new weirdness.

For me, not being a full time web site developer, it was near to impossible to keep track of all changes and concepts. It was especially hard to know which concepts to use for good compatibility with future versions of Plone.

New things arrived all the time - one example being the way you were supposed to make skins (the graphic appearance). It was completely revamped several times, each time introducing new layers of abstraction and complexity. Stuff like the stylesheet registry, where you are supposed to write XML code that registers your stylesheets in a magic registry that's accessible only via the magic web interface (the ZMI). Argl!

Trying to manage the way installations are made became more and more complicated, being a python script to begin with, then growing into countless XML files per product, making it very hard to keep track of what's going into what file. Argl, again!

No, I'm all happy I left Plone, and the learning curve is just one of the reasons.

(Beautiful explosion images from PD Photo.org)