Although one could have guessed otherwise, I’ve always been a happy Subversion user. Which former CVS user hasn’t? But now I’m ready to take the next step. For some of my personal stuff I’ve been using Mercurial for more than a year, so I thought it was time to take the plunge and switch my JSysTest project to Mercurial.

Google Code recently added Mercurial hosting, so I didn’t have to switch the hosting provider. The conversion process is quite simple: There are good instructions available that worked without any problems. From what I’ve seen so far, all functionality I care about is available (basically everything I used Subversion for).

One thing surprised me though: Converting a two dozen revisions took roughly 20 minutes, so for larger repositories you will have to take a long coffee break.

Recently, I had the opportunity to work with Subversion 1.5 on a medium sized project. Since there were more than 20 developers working on the project with some of them in a different country, there was no other way than to use feature branches extensively. We thought Subversion’s merge tracking would reduce typical merging errors by simplifying the process. But, well, we were wrong.

The sad truth is that few developers know exactly what their revision control system does behind their backs when merging branches. With Subversion prior to 1.5, things were pretty simple and there’s little magic involved. You have to be careful to state the correct revisions and if you don’t, you can still roll back your changes.

With 1.5, as long as everyone plays by the rules and sticks to the basic use cases, everything is fine. But if things go wrong (and they will), you’re really screwed. In a large team, there will always be someone who uses a buggy Subversion client, or someone who accidentally deletes merge properties (there’s often a high stress level before releases). And as soon as this has happened, merges may leave out revisions (or merge revisions twice) and you need in-depth knowledge to repair the damage. In some cases, even rolling back changes on trunk wasn’t enough if someone created a feature branch at the wrong time.

In a project consisting mostly of source files in a compiled language like Java or C#, obvious mistakes typically show up early. If you have lots of HTML, CSS, and XML configuration, or you’re working with an interpreted language, you don’t have this safety net. Your QA department (if you’re lucky enough to have one) will be the first to discover the problems.

So, if you’re a mostly happy Subversion 1.4 user and you’re considering migration to 1.5 or later for merge tracking, then my advice is: Don’t bother. Maybe recent versions fixed some of the problems, but I wouldn’t take chances. You don’t want to spend your nights fixing your source repository.

Externals definitions are a little known but very useful feature of subversion. Using an externals definition, you can create links to different parts of the repository (or even other repositories). Subversion clients then automatically check out the linked content into your working copy.

How this works is best explained with an example. Suppose you check out your code from the repository:

  svn co https://svn.your.host/project/trunk project

Inside your project, you need code from a different location in the repository or maybe even from some other repository. This could be a software module, test data, a generic part of your company’s standard build system or other things shared between projects. Many people do this manually:

  cd project
  svn co https://svn.your.host/whatever/tags/release-1.3 whatever

That’s tedious and error prone. A better approach is to use an externals definition that tells subversion to check out that project automatically for you. It works by adding some magic metadata to the project directory (that’s the dot on the svn command line!):

  cd project
  svn propset svn:externals \
    'whatever https://svn.your.host/whatever/tags/release-1.3' .

The svn:externals property accepts a set of declarations, each consisting of a directory name and a subversion URL. Because of this, the propedit command is more useful if you happen to need multiple declarations.

So, every time you check out (or update) the parent project, the whatever project will be checked out, too. The dependency to the other project is versioned by convention (I’ve linked to a tag in the example), but you can link to any URL and even include a revision number.

For more information see the Subversion book on svn:externals.

The Unix find(1) utility is a powerful tool, yet few people really understand how it works. It may be a bit confusing at first, but a programmer who knows his boolean algebra should be able to wrap his head around the basic concepts without much trouble.

You’ve probably seen simple find constructs like this:

find . -type f -name "*.c" -print

This prints all files (-type f) in the current working directory that end with .c (-name "*.c"). Easy. But what about this one:

find . \( -type d -name .svn -prune \) -o -print

Before you can fully understand this we’ll have to cover a few basics.

Find traverses a given list of directories and evaluates a user-provided expression for each file or directory it encounters. Like many programming languages, it uses short circuit evaluation for the expression. The catch is that the result of the expression is meaningless; no action is triggered if the result is true. Instead, operands like -print or -ls are ordinary predicates that always evaluate to true! Most predicates check if a file has a specified property (like if its name matches a regular expression), but -print is only executed for its side effect: It prints the file name every time it is evaluated.

Use the following commands to check if you understood the concepts:

find . -type f -print -print
find . -print -o -print

The first command prints the names of all files twice. That’s because the predicates are ANDed together. In the second command, the predicates are combined using the OR operator (-o in find syntax). Since the first -print already evaluates to true, there’s no point in evaluating the second one. So the file names are only printed once.

Let’s go back to the more advanced example:

find . \( -type d -name .svn -prune \) -o -print

Here we’ve got an expression consisting of two parts connected using the OR operator. The -print is only evaluated if the first part of the expression (the one in parenthesis) returns false.

Inside the parenthesis, there’s the -prune predicate we haven’t talked about yet. When -prune is evaluated, it has the side effect of skipping the whole directory tree. That means, after a -prune is evaluated, find doesn’t descend into the currently evaluated directory.

So, what does the command actually do? It recursively traverses a directory tree printing all files except .svn directories and their contents.

Once in a while, someone commits a revision that has to be rolled back later for some reason. Some teams do that as a matter of policy when people check in broken code that doesn’t compile or isn’t able to run all test cases successfully. Since Subversion (and similar revision control systems) are designed to never loose changes, that isn’t easily possible.

If a commit really has to disappear without traces for some reason, the only way is to import your repository into a new one leaving out the offending commit. That’s a rather drastic measure causing all kinds of problems with existing working copies, so use it with great care.

The approach used more often is to commit a new revision that takes back earlier changes. This is like undoing all changes manually and committing them in the usual way. The offending changes are still in the history, but don’t do harm anymore.

This is how it works to roll back changes made in revision N:

svn merge -c -N .

Note the minus character in front of N. This is the same as the following command (which also works with older subversion clients):

svn merge -r N:N-1 .

Afterwards examine the changes made to the working copy and commit them.

To understand what’s happening here requires some background on what the merge command does conceptionally: It records which changes have to be made to revision N to turn it into revision N+1 (called a delta) and applies it to the working copy. That isn’t what we want in our scenario, so we reverse the usual order of revisions on the command line. Effectively, we tell the merge command to create a delta the other way round. The result is a "rewinding" of the code base.

A lot of people use SSH to log into remote hosts. SSH is secure and works well, but if you have to access many hosts with long, well-chosen passwords there is a lot of typing to do just for authentication.

In this article I’ll walk you through a basic public key authentication setup. Password authentication may be more frequently used, but public key authentication is very convenient after the initial work is done. OpenSSH is used in the examples below, but the same process works much the same on other systems (for example putty/pageant on Windows).

In a nutshell, we’ll generate a public/private key pair on a local account and install the public key in all remote accounts we want to access. Then we can log into the accounts using the private key’s password instead of the remote account’s password. To avoid having to type the password multiple times, we’ll use a key agent which keeps the password in memory.

First of all, we generate a public/private key pair using ssh-keygen:

  $ ssh-keygen
  Generating public/private rsa key pair.
  Enter file in which to save the key (/home/matthias/.ssh/id_rsa):
  Enter passphrase (empty for no passphrase):
  Enter same passphrase again:
  Your identification has been saved in /home/matthias/.ssh/id_rsa.
  Your public key has been saved in /home/matthias/.ssh/id_rsa.pub.
  The key fingerprint is:
  6b:3c:60:9a:84:5a:06:59:62:f4:6b:25:d3:5b:0e:67 matthias@alanis
  $

Accept the default filename and then pick a strong but memorizable passphrase for your private key. If you ever forget the passphrase there is no way to recover it and the key will be useless. Don’t generate keys without a passphrase: An attacker breaking into your local account would then be able to log into all your remote accounts, too.

You can later change the passphrase using ssh-keygen’s -p switch. There are a lot of other command line options, but the defaults are usually just fine.

Now that the key pair is generated, we install the public key (from the id_dsa.pub file) in all remote accounts we want to log into conveniently. The private key in id_dsa has to be kept secret but needs to stay in your local ~/.ssh directory. Installing the public key works by appending it to the remote ~/.ssh/authorized_keys files. This can be done manually or using the ssh-copy-id tool:

  $ ssh-copy-id remoteuser@example.com
  [...]
  $

The tool prompts for the remote account’s password and then copies the public key. After this, you should be able to log in:

  $ ssh remoteuser@example.com
  Enter passphrase for key '/home/matthias/.ssh/id_dsa':
  [...]

Note that this time we’re being asked for the private key’s password, so the basic setup is done.

Typing the private key’s password multiple times (once per account) is tedious, so we want to enter it only once per local session. The ssh-agent and ssh-add tools shipped with OpenSSH can help here. The ssh-agent program runs as a per-user daemon in background. If the SSH_AGENT_PID environment variable is set then the agent is already running – which is pretty common on modern Linux distributions. Otherwise you can add the call to ssh-agent to your shell profile (eg. ~/.profile) or an X session startup script:

  eval $(ssh-agent -s)

As soon as the agent is running, you can use it from your local interactive session to register a private key. To do this you have to execute the ssh-add utility:

  $ ssh-add
  Enter passphrase for /home/matthias/.ssh/id_dsa:
  Identity added: /home/matthias/.ssh/id_dsa (/home/matthias/.ssh/id_dsa)
  $

After that, you can log into your remote accounts without having to enter the password.

I’ve been a happy user of RCS for years (in fact, my software engineering course at university was held by Walter Tichy, the original author of RCS). The good old Revision Control System may be a dinosaur, but it served me well for my configuration files and scripts.

Over the last few years, however, new and fancy revision control systems have entered the scene. My current choice is Mercurial, because it’s easy to use, popular, and well documented.

Since I needed some data to experiment with, I decided to switch my old RCS files over to Mercurial. Of course I wanted to keep the revision history, so a migration path was needed. For the conversion I used Tailor, a tool for converting between various version control systems. RCS is a bit dated, so it isn’t supported directly. But since the RCS file format is used in CVS, too, the natural upgrade path lead me to CVS and from there to Mercurial. This article shows how the conversion can be done.

So, first of all, we create a fresh local CVS repository somewhere:

    cvs -d /var/tmp/cvsroot init

The next step is to create a module inside the repository (scripts in this case), to create the desired directory structure (none in the example) and copy the RCS files there. The RCS files shouldn’t be locked as this could cause trouble.

    cd /var/tmp/cvsroot
    mkdir scripts
    cd scripts
    cp /path/to/repo/servehttp,v .
    cp /path/to/repo/b,v .

If you have a large number of files, it might be a good idea to automatize this using a script. Note that usually it’s no good idea to manipulate a CVS repository manually, but this is only for conversion. To make sure everything worked, we can check out the new module:

    cvs -d :local:/var/tmp/cvsroot co scripts

If everything worked as expected, we have a fully functional CVS repository that can be converted using Tailor.

Unfortunately, tailor-0.9.28 doesn’t work together with Mercurial 0.9.4, which is exactly the situation you find on Ubuntu 7.10. This incompatibility has already been fixed in tailor-0.9.30 and fortunately, you don’t have to install the new version. Extracting the tarball and executing the script from there works fine.

    python tailor -v --source-kind cvs --target-kind hg
      --repository :local:/var/tmp/cvsroot
      --module scripts --start-revision INITIAL > scripts.tailor

This creates a configuration file named scripts.tailor that has to be adjusted manually before the actual conversion. According to the documentation the purpose of the call above is to generate a template configuration file.

The configuration file should look like this:

    [DEFAULT]
    verbose = True

    [project]
    target = hg:target
    start-revision = INITIAL
    root-directory = /tmp/tailor-0.9.30
    state-file = tailor.state
    source = cvs:source
    subdir = scripts
    patch-name-format =

    [hg:target]

    [cvs:source]
    module = scripts
    repository = :local:/var/tmp/cvsroot

The defaults should mostly be finde, all I changed was the subdir setting and the patch-name-format to get better revision logs. More information about this can be found in the Mercurial wiki.

After adjusting the configuration it’s time to actually create the new Mercurial repository (this may take a while):

    python tailor --configfile scripts.tailor

We change to the created repository and make sure all revisions have been migrated:

    cd scripts
    hg log

That’s it. You can remove the CVS directory as soon as you’re confident that everything worked as expected. As a last step you might want to update the .hgignore file.

In a previous article, I discussed sshfs, a user mode filesystem based on FUSE for accessing remote files. Another useful filesystem type is encfs which provides transparent encryption of directories. With encfs, files are stored encrypted in a special directory under encfs’ control. The encryption algorithm and password are specified during creation of this directory.

To use encfs, you have to be a member of the fuse group and the fuse kernel module has to be loaded. See the sshfs article for more information. If you’re using Ubuntu or some other Debian-based distribution, the encfs package itself can be installed using the following command.

  sudo apt-get install encfs

After this, you can create your first encrypted directory. A useful convention to follow is to use a mount point somewhere in your home directory (e.g. ~/secret) and a corresponding directory for the raw encrypted files, starting with a dot (e.g. ~/.secret).

Execute the following command for setting things up (no root privileges required):

  encfs ~/.secret ~/secret

Note that you have to use absolute file names. The program prompts you for various cryptographic parameters and a password. After that, the ~/secret directory is mounted and ready for use. Try to create a few files there and see how the corresponding encrypted files appear in ~/.secret.

As soon as you’re done, umount the directory:

  fusermount -u ~/secret

The next time you want to use the directory, issue the same command as above:

  encfs ~/.secret ~/secret

Encfs detects that the directory is initialized already, prompts for the password and mounts the directory.

In contrast to other approaches like dm-crypt, encfs directories grow and shrink on demand. After all, the files are encrypted individually. The downside of this approach is that attackers can draw conclusions based on the number and size of files in the encrypted directory. Filenames are encrypted, too, but depending on your use case, the amount of privacy encfs provides may or may not be enough.

For mounting directories from a remote machine, usually NFS or SMB/CIFS is used which requires a running file server. On many linux servers, however, all you have is a shell account, so file transfer has to be done via scp.

Using the FUSE Linux kernel module, which provides user space filesystems and its sshfs client, you can mount remote directories from machines where all you have is an SSH account. If you can log in via SSH you can also mount the directory, there is no need to modify or configure anything on the remote machine.

On the client machine, you have to install FUSE and sshfs. If you’re running Ubuntu, execute the following command:

  sudo apt-get install sshfs

To get the permissions to use FUSE on your local machine, your user has to belong to the fuse group. This can be achieved using vigr(8), for example. Don’t forget to log out and in again for the change to take effect. You can see which groups you are part of using the groups or id commands. From this point on, you can work without root privileges.

The next step is to create a local mount point on the client machine (using the hostname of the remote machine would be a good convention to follow):

  mkdir mountpoint

Now you can mount the remote directory:

  sshfs remote_username@remote_hostname:/home/media mountpoint

This mounts the /home/media directory on the remote server. If you want to mount your remote home directory and the user names on the local and remote system are the same, the command can be even shorter:

  sshfs remote_hostname: mountpoint

When you’re done, you can umount the directory:

  fusermount -u mountpoint

Check the sshfs(1) manpage for more information, like mapping user IDs or performance tuning.

This week at work I had the opportunity to build an RPM for some third-party software package.

My previous experiences with RPM date back to SuSE 5 almost ten years ago and I haven’t touched it ever since. Packaging for Debian or even NetBSD is mostly a painless process, so I hoped today’s RPM wouldn’t be much different. Things worked out fine, except for one thing: RPM uses a well-known base directory (/usr/src/packages on SuSE) which only root can write to. Sure, you can pass ownership to another user, but without root access in the first place you’re out of luck.

Using ~/.rpmmacros, users can define their own private staging area, but that’s no elegant solution either. Not even The RPM Guide came up with a usable receipe for this.

After some unsuccessful experiments, I decided to have a look at how python’s distutils solve this problem. With distutils, you can build RPMs from python packages (python setup.py bdist_rpm) without superuser privileges. That’s why I figured it had to be possible.

Distutils uses rpmbuild’s --define switch (undocumented in the man page) and the _topdir macro:

  rpmbuild -ba --define "_topdir /path/to/staging_dir"
     /path/to/staging_dir/SPECS/package.spec

Note that /path/to/staging_dir has to be an absolute path, leading to the directory which contains the same directory structure /usr/src/packages has. `pwd` can be used to avoid hardcoding the path.

From a script, I create the directory structure in my local working directory, copy source tarball and spec file to SOURCES and SPECS and execute rpmbuild.

The created RPM can then be found in the RPMS directory.