Blog Entries tagged open source
Feeds: RSS | Atom

Continous Integration with Hudson - embarrasingly simple!

Published: 2011-01-27 19:24 UTC. Tags: open source software testing

I'm working on a rather large reporting and analytics application that runs on top of Hadoop at work. It has tests. A whole bunch of them, actually. That's good.

So far, we've been running the tests manually when making new releases. But doing it more often is always better, since it gives you an indication on when things went wrong, and also forces you to keep your tests in a state where they pass. Some people call it Continous Integration.

Now, you can do all the work getting your builds to build and run tests yourself, via cron and scripts and other types of messiness. Or you can try an existing solution. Today I decided to try Hudson.

That turned out to be embarrasingly simple to get started with. Basically, it's a matter of:

  1. Downloading hudson.war from their site.
  2. Start it by running java -jar hudson.war
  3. Go to http://localhost:8080 with a web browser of your choice. That would be Opera in my case. You have to eat your own dog-food.
  4. Go to the Hudson management screen and enable the git plugin
  5. Setup a new project. Tell it where the code is and on which branch.
  6. Configure what commands to run to build and test. Make the test command output an xunit xml file.
  7. Tell Hudson where that xml file is.

Result: Hudson will periodically poll git and run my build and test commands, then show a changelog and what tests failed. All this after 30 minutes of setup time. I'm impressed.

0 comments.

Whenever You Need a Random Password

Published: 2010-04-14 20:28 UTC. Tags: open source

apt-get install pwgen

0 comments.

Dualhead at home

Published: 2009-02-04 19:27 UTC. Tags: open source hardware linux syncmaster_204b
Dualhead setup

I upgraded my home workstation to Ubuntu 8.10 the other day. I was kind of worried that this would cause trouble with my X Window System setup. I've had major trouble with my Samsung Syncmaster 204b before, with its faulty DVI implementation being the major source of trouble.

My worries were well justified - no image on DVI after reboot. Tried connecting it via VGA, and that kind of worked - a slight fuzziness, which almost went away after pressing the auto-adjust button on the monitor.

I got tired of the monitor and decided to go for a new one - so I ordered a Benq FP241W 24" WIDE TFT. It's using the less common MVA display technology instead of the more common TN. This is supposed to mean that it's colours and viewing angle are superior to standard panel.

Thinking about it, I came to the conclusion that I might be able to stand a slight fuzziness on my 20" samsung if using it as a secondary panel in a dualhead setup. So, when the new panel arrived, I connected the 20" Samsung via VGA, and the new monitor via DVI.

With my usual luck, I got the same problems with the new panel as with the old one - problems syncing the DVI signal, with the result being that the panel went black.

However, after switching from the open source ATI driver to the proprietary fglrx driver, things started working, and using amdcccle, the ATI driver control program, setting up dualhead was really easy.

The result - a resolution of 3840x1200 at home. That's even slightly better than at work, where Opera Software have equipped me with a 19"+24" combo.

I'm not looking forward to the next Ubuntu upgrade, though. Sometimes, I just hate the X window system.

1 comments.

ViewVC Django Integration

Published: 2008-10-25 20:14 UTC. Tags: open source software django version_control

I've written a Django product which helps integrating ViewVC in a Django Site.

0 comments.

Some documentation on the sourceforge to roundup converter now available

Published: 2007-08-27 21:56 UTC. Tags: open source

I had one comment on my last post asking for information about the converter from sourceforge to roundup.

I wrote up a small README file on the three different converters that are available. The README is available in python's subversion repository:

http://svn.python.org/projects/tracker/importer/README.rst
0 comments.

bugs.python.org finally in production!

Published: 2007-08-25 22:27 UTC. Tags: open source software

For over a year, I've been working on replacing the bug tracker used by the python project. Earlier, they used the horrible bug tracker provided by sourceforge. Now, since last thursday, they are using their own tracker based on roundup.

Hooray! I'm very happy we're finally there!

The new tracker is at http://bugs.python.org

Earlier posts on the subject:

3 comments.

PlacelessTranslationService bit my head off!

Published: 2007-08-09 16:34 UTC. Tags: open source software plone
Today, I made the mistake of adding a <myproductname>-sv.po file in the i18n directory of my product, which accidentally were marked with
Language: en

This of course made the english strings appear in swedish, but that's only to expect. What happened then was worse - after correcting the file to contain:

Language: sv

..there were no change! The english strings still appeared in swedish!

After a frustrated debug session, it turns out that the language property of the GettextMessageCatalog object stored in the ZODB (I think) is not reloaded when the Language property in the file is changed. So, even if you change the Language line in the .po file, the translation will still be marked as being in the language it first was entered as.

The solution? Move the .po-file out of the i18n directory, restart Zope, then move it back, and restart again.

Gah!

Well, at least I can be glad that my CMS is an open source product, allowing me to debug problems properly.


4 comments.

Fighting tracker spam with SpamBayes

Published: 2007-07-28 00:43 UTC. Tags: open source software

During the last few days I've had time to do some programming just for fun. When you combine vacation and bad weather, you can be very productive :-).

Most of the time, I've been doing work for the roundup tracker instance for the python development team. I wrote about the new importer earlier, and now I've also created an anti-spam system based on SpamBayes.

For those interested, there's a technical description of the roundup spambayes integration in the roundup wiki.

SpamBayes seems to be a nice piece of software, especially now when it has an XMLRPC interface. Imagine having a SpamBayes XMLRPC server on your network, and then a plugin in your mail user agents that calls it to rate messages before sorting them down to folders, and a button in the user interface that allows users to report messages as being spam or legitimate content. That would be very powerful and give very few incorrect ratings as each organisation's SpamBayes server would learn what's legitimate content for the organization where it's installed.

When to sort?

As all users receiving large amounts of mail (in my case mostly because of mailing list subscriptions), I sort my mail. For my personal mail, I let the Cyrus IMAP server sort whenever the messages arrive, using the Sieve sorting language. I use the Sieve filter interface in Squirrelmail to create rules. This is very convenient as the mail is always sorted when I arrive at my mail client, regardless of which client I use. Also, I only have to define my filter rules at one place. I only wish there were more mail user agents with decent Sieve filter support.

For sorting out spam however, the later the sort is done, the better, at least for some kinds of anti-spam measures, including statistical filters such as SpamBayes. Imagine the following scenario:
  • User A, who likes to arrive at the office early in the morning, opens his INBOX and find 2 messages that have been incorrectly classified as legitimate mail. He presses the 'report as spam button', which in an ideal world will teach the local SpamBayes server to score the message better.
  • User B, who is a lazy bastard, arrives two hours later. When his mail user agent sorts his mail using the local SpamBayes server, he can benefit from the work made by User A earlier in the morning, as the two messages are now correctly sorted.
On the other hand, the statistical filters seems to work well enough for the typical spam that sorting early (on the server) is probably good enough, so I think I'll continue to let my Cyrus IMAP do the sorting for me. A button in my mail user agent for classifying the message would still be neat, though.
0 comments.

Life as a sf.net conversion script author

Published: 2007-07-25 13:05 UTC. Tags: open source software

About a year ago, the infrastructure team of the python language project sent out a call for trackers. They had come to the conclusion that the tracker available at sourceforge was not good enough. I can understand that - it's very hard to use, and since it's running on sourceforge's servers, it can't be customized.


I and several other people thought that roundup, a tracker infrastructure would be a good choice, so we formed a team and managed to come up with a submission for the call. This included writing a conversion script that took the data from sourceforge and imported it into  the new tracker. I created this script based on a screenscraper library for sourceforge written by Fredrik Lundh. This was importer #1.

Later on, roundup was selected as one of the two final alternatives. Happy happy, joy joy :-). A team was formed (including me) for creating the tracker, and Upfront Systems kindly provided a linux host for running the tracker.

Now began the real work of designing the tracker and adjusting the importer to the final schema. During this time, sourceforge managed to fix their broken xml export, so I wrote a new importer that instead of screenscraping webpages took an xml file as input which was much faster and more reliable. That is, I wrote importer  #2.

Later on, when we were beginning to get ready for production launch, a real showstopper shows up - the xml export from sourceforge couldn't cope with the size of the python project - the export was missing data.

After several months of waiting for sourceforge, they have a new export script that includes all data. Unfortunately, it has a completely new xml format. Writing a third importer was less than fun, but I managed to complete importer #3 yesterday. Hopefully, I didn't introduce that many bugs..

Who knows, maybe the python project will have a new tracker sometime this year? :-)

Try out the new tracker at http://bugs.python.org.

0 comments.

Why doesn't normal people find Open Source Software?

Published: 2006-11-07 21:01 UTC. Tags: open source software

I spoke to a customer today. He's head of the IT department for a middle-sized organization with 6000 users.

He had a project going on with the goal of finding and implementing a document management system for the whole organization.

The project group had reported back to him telling him that they had found three (yes, 3) candidates, and that the candidate they thought was best was some Microsoft product.

He was a little bit puzzled that they only had found three candidates. I had to agree, so I immediately made a quick search on Freshmeat for document management and got 105 hits.

I think this reflects a problem that the open source world has to pay more attention to - people don't know that there are alternatives to the products from Microsoft and the other big companies with large PR budgets.

0 comments.

SuSE-linux

Published: 2006-10-17 21:14 UTC. Tags: open source software humor in-swedish

SuSE är en konstig Linuxdistribution. De älskar att göra allting annorlunda och gärna på ett sätt som ställer till det för oss som vet hur man gör på alla andra distributioner. Don't get me started när det gäller deras kreativa tolkning av hur PAM och NSS ska fungera.

Det har inte blivit bättre sedan de blev uppköpta av Novell...

Nu har jag kommit på varför den är knäpp. Det hörs ju på namnet!

Sosselinux

0 comments.