A few weeks ago I wanted to enable digest authentication on an Apache 2.2 web server. I got basic authentication working within minutes, but I didn’t want to send plain text passwords over the web, so how difficult could it be?

I followed some tutorial on the net but despite following the instructions throughly, all I could get was an error message in my error.log file:

Invalid command 'AuthDigestFile', perhaps misspelled or defined
  by a module not included in the server configuration

It turns out that the AuthDigestFile configuration command from Apache 2.0 and earlier (see 1.3 docs or 2.2 docs) doesn’t work on Apache 2.2 anymore. In 2.2, you use AuthUserFile, just like with basic authentication.

It’s all in mod_auth_digest’s documentation, including a nice, working example. It didn’t occur to me until late that the tutorial I had found on the net could have been outdated. Oh well.

Usually, I download all my email via fetchmail, sort it into folders using procmail and read the mails with mutt. This works reasonably well and I’ve been using this mail setup without major changes for about 10 years. Since I recently bought a netbook, I also wanted to read mails with it, but without touching my existing mail setup. The netbook should only read mails directly on the server without downloading anything to local disk.

My mail service provider 1&1 supports IMAP and a webmail client that I sometimes use, so I decided to follow the webmail client’s conventions regarding folder names. This way, I can use mutt on the netbook and the webmail client interchangeably.

Based on a mutt configuration for gmail, I ended up with the following IMAP-related settings:

set from = "your.address@example.invalid"
set realname = "Your Real Name"

set imap_user = "matthias@mafr.de"
set imap_pass = "*******"

set smtp_url = "smtp://$imap_user@smtp.1und1.de:587/"
set smtp_pass = $imap_pass

set folder = "{imap.1und1.de/ssl}"
set spoolfile = "+INBOX"
set record = "+Gesendete Objekte"
set postponed = "+Entwürfe"
set trash = "+Papierkorb"

set move = no

mailboxes "+INBOX"

set header_cache = "~/.cache/mutt/headers"

The most important account settings are imap_user, imap_pass (optional, mutt will prompt for it), and folder. Folder names prefixed with a + sign are relative to folder. Note that I disabled moving read mails to the mbox folder; I want to keep everything in my inbox. Because of the record setting, all sent mails are recorded in a folder on the IMAP server.

To speed things up, I enabled the header cache. You could also cache entire messages, but since mutt doesn’t expire them yet, I don’t use this feature.

What doesn’t work: Neither fetchmail nor the 1&1 server propagate the IMAP "seen" flag of the emails to a BSD mbox-style "Status: RO" header. Because of this, I have to mark all downloaded mails as read manually.

Once in a while, I install Python packages from source using distutils. The distutils-powered setup.py script found in many packages installs software in /usr/local/ – quite useful because it doesn’t interfere with packages managed by your distribution. Since Ubuntu Jaunty and Python 2.6, this doesn’t work anymore.

Previously, your usual command python setup.py install installed packages in /usr/local/lib/pythonX.Y/site-packages/, depending on your Python version. As of Ubuntu Jaunty, Python 2.6 no longer has this directory in sys.path. Instead, it expects local packages in /usr/local/lib/python2.6/dist-packages/, so your locally installed packages are no longer found.

I learned from this posting that distutils now supports an --install-layout=deb switch that installs the package in /usr/local/lib/python2.6/dist-packages. The switch is accepted in distutils from Python 2.4 and 2.5, too, without having any effect, so you can always include it in your command line.

To sum things up, the new command line for installing python packages in /usr/local/ is

  python setup.py install --install-layout=deb

This probably applies to Debian systems as well, judging from the name choice of --install-layout’s parameter. I updated my posting on stow to reflect this change.

Upgrading from Ubuntu “Hardy Heron” to “Intrepid Ibex” wasn’t the smooth ride I was used to. I’m a pretty happy Ubuntu user and my system (an old Acer Travelmate 291) has been running all Ubuntu releases since “Breezy Badger” without a single reinstall. This time, however, I experienced problems with WLAN, sound, battery status display, and postfix SASL authentification.

Since “Hardy” is a “long term support” release, the update manager doesn’t announce the availability of the “Intrepid” release. You first have to go to “System / Administration / Software Sources” and in the “Updates” tab change “Long term support releases only” to “Normal releases”. Then the update manager will display the “Update” button.

After the update, kernel 2.6.25 was installed. Since WLAN and sound didn’t work, I upgraded to 2.6.27 which was also available but things didn’t improve. A look at dmesg(1) showed that the firmware for my ipw2100 chip wasn’t found. I installed the linux-firmware package and things worked again.

Sending mail didn’t work because postfix wasn’t able to authenticate to my provider’s SMTP server. Nothing in the logs pointed to the exact problem, but after re-generating all .db-Files in /etc/postfix/ using sudo postmap hash:filename mysteriously solved the problem.

Maybe my system is showing its age and has accumulated too much cruft over the years. Or maybe I shouldn’t update my system so soon after a new release …

Virtually everybody agrees that backups are a good idea, but few people actually do them. Backup software is often time consuming to set up or just overkill for a single-user system. In this article I’ll show how easy it is to build your own backup solution on Linux using rsync and an external disk drive.

The backup schema proposed here is based on Mike Rubel’s excellent Easy Automated Snapshot-Style Backups with Linux and Rsync article. In my schema we leave out anything sophisticated and concentrate on the single-user scenario. The result is probably one of the simplest and least expensive backup systems you can get away with.

Features

These are the features in a nutshell:

• No network, server or special software (except rsync) required
• Suitable for single-user workstations
• After setup usable without root access
• Multiple backups are possible, with each being a complete snapshot
• Restoring a backup is done using standard Unix utilities (cp!)
• Very space-efficient due to use of hard links between backups
• Backup mediums are cheap (any external USB drive will do)

Rsync snapshots make use of Unix hard links to save space. This works under the assumption that only a small percentage of files change between backups which should be true in many cases.

The backup medium shouldn’t be mounted all the time because that would increase the risk of accidental overwrites and other hazards. An external drive disconnected from both power and USB most of the time is a pretty good solution for single-user workstations. The downside is that backups have to be triggered manually which is a bit inconvenient and requires discipline.

If you need networked multi-user backups, try dirvish which shares many advantages with the solution presented here. It is a proven rsync-based system and can be executed from a central backup server via cron.

Setup

First of all you need an external disk drive. I would suggest that it should be at least 2-3 times as big as the data you want to backup. Then create a file system that supports hard links (ext2/3 do, among others). When using ext2/3 it’s a good idea to set the usual 5% space reservation for user root to zero; otherwise you’d waste a lot of space. You might also want to adjust the file system check interval using tune2fs(8).

Make sure your user can mount the disk and write to it. Create a dedicated backup directory on your disk, especially if you use the disk for other archiving purposes, too. It also helps to easily check if the disk is mounted.

Process

In your backup directory, each snapshot gets its own subdirectory. The snapshots share common, unmodified files to save space. After a few backups, the directory looks like this:

  $ ls /path/to/backup/dir/
  backup.0 backup.1 backup.2 backup.3
  $

The latest backup is always backup.0, the oldest is the one with the highest number (backup.3 here). When a new backup is made, the oldest directory is deleted and the other ones’ IDs are moved up. That means, backup.2 becomes backup.3, backup.1 becomes backup.2 and so on.

When a new backup is made, rsync links to files in backup.1 if a file hasn’t changed since last time. The rsync command line responsible for this behavior looks like this:

  $ cd /path/to/backup/dir/
  $ rsync --archive --link-dest=../backup.1 SOURCE_DIR/ backup.0

Note that the trailing slash with SOURCE_DIR is significant.

In case you don’t want to backup certain files (browser caches or the ~/.gvfs mountpoint come to mind), you can add exclusion patterns to rsync’s command line. Using the --filter switch (or the more limited --exclude-from), those patterns may be specified in a separate file, too.

Putting all together, you should end up with a simple backup script and optionally an rsync filter file.

Recently, I decided to re-partition my hard disk because I needed a larger swap partition. In this article, I’ll describe the process of backing up an existing system and restore it somewhere else. I haven’t done anything like it in years, so I was up to some smaller surprises that I was fortunately able to solve quickly. In any case, being extra careful and planning ahead usually saves quite a few headaches. Moving a system is no trivial task, so follow this article at your own risk.

First of all, the existing data has to be backed up on another system. The backup should be created in single user mode to make sure there are no processes running that modify the filesystem. Since there are a few things you don’t want to backup (like /proc or other pseudo filesystems), you should figure out what they are, maybe using mount(8). For my system, this command worked:

  tar cf -
      --exclude /proc --exclude /sys --exclude /var/run
      --exclude /var/lock --exclude /dev
      /
      | ssh user@host "cat > backup.tar"

The command creates a tarball (excluding a few directories) and directly sends it to a different host. If you have a second disk in your computer with enough space then you could also save it there, but a backup on a different system is always recommended. It’s also a good idea to test the backup to see if everything is there and files can be extracted without errors.

After the backup is done, boot into a rescue system (like the Ubuntu installation CD) and re-partition the disk. In my case, an alternative would have been to shrink my root filesystem using resize2fs(8), to adjust the partition’s size (via fdisk(8)) and add a second swap area at the then free area. I decided against this because if I made any errors during the procedure I wouldn’t find out until the filesystem fills up (and possibly overwrites part of the swap area).

So I created a new swap partition (type 82) and a root partition (type 83) and formatted the partitions using mkswap(8) and mkfs. Make sure you don’t lose the generated UUIDs – you’re going to need them later. If you’re using ext2 or ext3 use tune2fs(8) for the usual adjustments (reserved block count, file system check interval etc.).

Mount the new root filesystem (/mnt is used in this example) and extract the backup there:

  cd /mnt
  ssh user@host "cat backup.tar" | tar xpf -

Make sure everything looks OK and don’t forget to create the directories that were excluded from the backup:

  mkdir /proc /sys /var/run /var/lock /dev

Since newer distributions use UUIDs to identify filesystems, we’ll have to adjust the extracted system to use the newly generated UUIDs. Those UUIDs hide in several places:

• /mnt/etc/fstab
• /mnt/boot/grub/menu.lst
• /mnt/etc/initramfs-tools/conf.d/resume (for hibernation)

Make sure to update the UUIDs everywhere. I didn’t get them all at the first attempt, so I got kernel panics and hibernation didn’t work. Then setup the master boot record using grub. I did it at the grub shell, but the following command should work, too:

  grub-install --root-directory /mnt

After booting into your system for the first time, you have to run update-initramfs -u and then restart the system. Otherwise hibernation will be broken. After that’s been done, your system should work the same way as before.

When I wanted to suspend my notebook today the hibernate button was gone from the menu. Running the /etc/acpi/hibernate.sh shell script as root still worked, but there was no way to get that silly button back.

My first idea was some kind of configuration trouble, suddenly appearing for no reason (I never trusted gconf completely anyway). But obviously, that wasn’t the problem:

 $ gconftool-2 -g /apps/gnome-power-manager/general/can_hibernate
 true
 $

After some time reading shell scripts I found the problem tough. A while ago I added more RAM to my notebook and also had to configure a swap file because my swap partition was too small. Things worked fine for a while, but then an upgrade of package powermanagement-interface (0.3.16) came along with the following change:

powermanagement-interface (0.3.16) gutsy; urgency=low

  * pmi.acpi: Disable suspend and hibernate if /host is mounted via fuse;
    disable hibernate if a swap file is active.

I cannot say if there could possibly be problems with swap files, but this change leaves me with two ugly options:

1. Hack the /usr/sbin/pmi shell script to disable the check (gah!)
2. Render the swap file obsolete by re-partitioning my disk

Unless someone comes up with a better idea, I’m leaning towards option 2 …

Package management tools like apt-get make it very easy to install and remove software. The package manager takes care of downloading the package together with its dependencies if, of course, someone has taken the time to create a package and made it available to you. In some cases, you don’t want to interfere with your local package management, however. With a bit of bad luck, third party packages can mess up your package system or cause conflicts when upgrading your distribution.

Because of this, people often install source packages to /usr/local without bothering to create deb or rpm packages. The problem with this approach is that it leads to a lot of clutter in /usr/local and it makes uninstalling software pretty difficult. Using the stow tool, you can easily install software from source and uninstall it later. Stow doesn’t even need a package database or configuration files for this.

The idea of stow is to install each software package into a directory structure on its own and use symlinks to make it available to the rest of the system. Suppose you want to install a nice little software package (let’s call it yourpackage-0.1) from source. You would then install it in /usr/local/stow/yourpackage-0.1, where it has its own directory structure with bin, lib, include and other directories. Stow would then create symlinks, so that /usr/local/bin/foo would be a symlink to /usr/local/stow/yourpackage-0.1/bin/foo.

If you’re using stow for the first time, you have to create the /usr/local/stow directory. It doesn’t have to be owned by root; on my system it belongs to the src user, which I use to install software.

Let’s install two software packages from source to see how it works. The first example is libdiscid, a library written in C which has autoconf support:

   $ tar xvzf libdiscid-0.1.tar.gz
   $ cd libdiscid-0.1
   $ ./configure --prefix=/usr/local/stow/libdiscid-0.1
   $ make
   $ make install
   # cd /usr/local/stow && stow libdiscid-0.1
   # ldconfig

Note the --prefix argument which tells configure where to install the package. The first five commands can be executed by the user owning /usr/local/stow. Only the creation of symlinks done by the stow tool requires root privileges. The ldconfig call is required for libraries to tell the dynamic linker about the new library.

Python packages work much the same way. We’ll use python-musicbrainz2 as a demo:

   $ tar xvzf python-musicbrainz2-0.4.0.tar.gz
   $ ./setup.py install \
        --prefix=/usr/local/stow/python-musicbrainz2-0.4.0 \
        --install-layout=deb
   # cd /usr/local/stow && stow python-musicbrainz2-0.4.0

Obviously, the --install-layout=deb parameter is only relevant for Debian-based systems.

Uninstalling a software package works using the following command:

   # cd /usr/local/stow && stow -D python-musicbrainz2-0.4.0

This only removes the symlinks but not the package itself. To get rid of it altogether, you have to remove the python-musicbrainz2 directory manually.

The a2ps command line utility converts text files to PostScript which can then be sent to a printer. The resulting output contains all kinds of boxes and additional information which limits the space on the printed page to make your own notes.

Of course, this can all be turned off, but it requires quite a few command line switches to get rid of it all. Fortunately, there is the (undocumented) lp user option which tells a2ps to print like a good old line printer. User options are pre-configured profiles which can be enabled manually.

To print an RFC with two pages per sheet using the lp user option, use the following command:

    a2ps -o output.ps -=lp -2 rfcXXXX.txt

You can then print the resulting output.ps file using lpr. If you leave out the -o switch and its argument, the output is sent to the printer directly. If you don’t have a PostScript enabled printer, your local printing system should take care of converting it to your printer’s native format.