Thunderbird & Ubuntu 10.04


Thunderbird 3 comes with Ubuntu 10.04, but sadly it doesn’t integrate with the default desktop as well as it could — it doesn’t link to the envelope icon in the Indicator Applet, and it uses its own popups instead of the libnotify mechanism Ubuntu prefers.

These are easily fixed, however …

http://ubuntuforums.org/showpost.php?p=9150987 tells us how to add Thunderbird to the menu, along with actions such as Compose New Message and Contacts (read the whole thread to see how Calendar would work if you had Lightning installed); but it doesn’t address how to get notifications working.

The experimental libnotify-mozilla XPI for Thunderbird itself, on the other hand, provides the notification of new mail — but if Thunderbird is not running, or if there are no new messages, nothing will be visible in the Indicator.
http://ubublogger.wordpress.com/2010/02/02/how-to-install-the-experimental-version-of-libnotify-mozilla/

A combination of these two still doesn’t produce a perfect solution, but it’s good enough until Ubuntu & Mozilla sort themselves out and get Thunderbird capable of replacing Evolution on the desktop fully.

The Indicator Applet showing Thunderbird status

Tags: , ,

Exim4 and authenticating from a database


The exim authenticators take care of collecting a username/password from the user during the SMTP conversation, and are often used to check against a database to see if they are ‘correct’. The syntax for this check gets a few people confused, so here’s a description …

Your database is storing an encrypted password I hope; if it is not you have other issues. You should also be following standards and prefixing the password with the name of the encryption scheme, so the DB value looks like :- {md5}CY9rzUYh03PK3k6DJie09g==

However, if you are simply storing the encrypted data without the scheme name (as I was until I started writing this!) you need to stick to a single scheme and hard-code this into the exim authentication config, which is a big source of confusion.

How it all works :-

The authentication session provides an actual password (it was lightly encoded with base64 during the SMTP session, but this has been taken care of automatically). Under PLAIN authentication, for example, $3 will contain the actual user password.

A DB query such as ${lookup mysql{...}} returns “encrypted-password” (not “{scheme}encrypted-password”)

Therefore in order to compare the password with the encrypted password, we have to say what encryption scheme the database is using — md5, sha1, whatever. We cannot decrypt what the database returns, so we must encrypt the password — but the difference is invisible to us, Exim does the right thing.

In order to encrypt the password, we need to tell crypteq which encryption scheme to use — “sha1″ normally. We need to tell crypteq explicitly what scheme we’re using, it cannot guess by looking at the values. Exim is expecting “{scheme}data”, but our DB isn’t returning that …

Also, crypteq’s *syntax* is confusing because it doesn’t read like “encrypt the password with ’sha1′ and compare that to what’s in the database”, it reads “compare the password with the ’sha1′-encrypted data from the database”.

And, for good measure, the way you say “sha1″ demands curly brackets, which are *not* part of the command syntax, and therefore need to have backslashes in front of them …

crypteq takes two strings, the unencrypted password in one, and the encrypted password PLUS the encryption scheme name in the second. We need to explicitly say \{sha1\} in front of the SQL lookup.

crypteq {$3}{\{sha1\}${lookup …}}

${if} takes three values; the query (the entire crypteq thing from above), and the two results.

server_condition = ${if crypteq {$3}{\{sha1\}${lookup …}}{yes}{no}}

read more about crypteq: http://www.exim.org/exim-html-4.40/doc/html/spec_11.html

Here’s a full authenticator for PLAIN …

begin authenticators

plain:
    driver = plaintext
    public_name = PLAIN
    server_advertise_condition = ${if eq{$tls_cipher}{}{no}{yes}}
    server_condition = ${if crypteq {$3}{${lookup sqlite {DB \
        select password from users where username = '$2';}}}{yes}{no}}
    server_set_id = $2

Tags:

Creating icons for Nagios


By default, the statusmap of Nagios does not include any useful icons for your equipment. There are some icon collections on http://www.monitoringexchange.org/, and a handy set can be installed with the nagios-images package.

However, there will come a time when you want your own customised icons, and information on how to produce them is hard to come by. The status map is assembled as nagios starts with the GD library, and therefore the recommended format for icons is .gd2, but strictly speaking that’s an intermediate format and most tools don’t give you any options to create them.

Start by creating an SVG files in Inkscape. Make the icon around 40×40 pixels; this small size won’t matter at all while editing in Inkscape, as vectors scale very well! Avoid using black — the next steps in the toolchain will treat black as transparent. Instead use a blackish colour, such as #010101.

Just before saving the file, go to “Document Properties” and set the page size to fit the selected icon.

When the SVG file is ready, save it. We’re finished with Inkscape now, and now we drop to the command-line to use rsvg-convert from the librsvg2-bin package. By default this will create a PNG file, which we can use as the Nagios icon_image. Then we use netpbm tools to grab the transparent part of the PNG, and write out a GD2 image

$ rsvg-convert example.svg > example.png
$ pngtopnm example.png | pnmtopng -transparent =rgb:00/00/00 2>/dev/null | pngtogd2 /proc/self/fd/0 example.gd2 0 1

The pngtogd2 command comes from libgd-tools, and is slightly annoying in that it won’t take STDIN … so we fake it using /proc/self/fd/0. The “0 1″ parameters set the chunk size and format, and are cargo-culted from the web and Nagios documentation. The black areas in the PNG are marked in GD2 as being transparent.

So now you can create nice Nagios icons easily. I’ll be publishing a set of basic icons and their source SVG files on code.inode.co.nz soon.

Tags:

Organising Nagios config files


Nagios is great, but I still get annoyed with the standard configuration setup. Hosts, Hostgroups, Services … whenever you add a new host, there are a lot of places to touch to get it configured properly.

Most customers think of their network in terms of “hosts” — they add a physical host, they run services on that host, they want to monitor that host. So how can we reduce the amount of config work to be done?

I’ve started to play with a scheme I’m calling ‘capabilities’. A capability is a combination of a hostgroup and a service definition, both with the same name. Here’s the “ssh” capability :-

### SSH {{{
define hostgroup{
    hostgroup_name      ssh
    alias               Hosts with ssh on TCP:22
}
define service{
    hostgroup_name      ssh
    service_description ssh
    check_command       check_ssh
    use                 service.template
}
# }}} ssh

All I have to do in order to have checks against this capability is to add the host to the hostgroup ’ssh’, and I can do this directly from the host object itself …

define host{
    host_name       benjamin
    alias           Mail Server
    address         10.11.12.13
    use             host.template
    hostgroups      ssh,smtp
}

No messing around with specifying this host anywhere else, all the references it needs in order to exist are in one file, and so are the complete set of references to the service checks that will be performed.

Now admittedly, this does add extra hostgroups to the CGI interface, and if you are using that on a regular basis you may not appreciate the extra clutter. However, for the sorts of thing I’m doing, it’s more important to be able to quickly and accurately describe a host & associated checks.

Tags:

The hacker emblem


One small piece of HTML and one small piece of CSS.

<pre class="hacker"> .<br/>..:</pre>
pre.hacker {
     letter-spacing: -2px;
     line-height: 6px;
     font-size: 10px;
     text-align: center;
}

Job done.

 .
..:

Tags:

userpkg — installing users from packages


If you’re looking after handfuls of computers belonging to armfuls of different companies, you end up going through hoops trying to get your own user(s) installed each time you want access to a new machine.

userpkg aims to make this simpler, by using Debian package management.

Once userpkg is installed on a machine, you can introduce your new admin users as packages. userpkg will take care of choosing an available username, generating a new password, installing ssh keys and setting up sudo rights.

If you have set up your own private repository, you can now automatically upgrade ssh keys across the population of customer machines …

The initial public release of userpkg can be found at http://userpkg.inode.co.nz/. It’s feature-limited at this stage, but will grow; faster if people suggest things to me!

Tags: ,

When is a bash script too big?


I quite like writing shell scripts, I’ve been doing it for years and often when I start to address an automation question I naturally start to think in terms of “automating the command-lines”.

However, some problems very rapidly run out of bash’s ability (or often, out of its readability). Bash and the associated tools I often use (grep, cut, sed, awk, sort and so on) are very line-oriented; it’s difficult to get them to make decisions based on any part of the input except the current line and any summary history you’re building up.

So, one reason to switch to a different language is where you need to grok a whole set of structured files in one go. For this I often go to perl, and read files directly into a hash of hashes (of hashes of …). Then I can walk through that structure in any order, rather than having to touch the input files again. It may eat memory, but these are only scripts we’re talking about, not programs.

Another reason to switch is less obvious, but as soon as you try to make your bash script look like it’s written in some “proper” functional language, you’re into a losing battle. For me, this generally happens around the 250+ lines mark; if the script is that long, it’s going to have a lot of structure. Bash can do functions, but it’s difficult trying to make variables act like they’re scoping properly, and as for passing parameters that might end up being executed … well, that becomes painful. You have to construct the original calling parameters with levels of quoting that explicitly need to know how much dereferencing they’re going to encounter.

An example; I have a script here that makes a lot of changes to important files, so I thought it would be a good idea to be able to log each command invocation. So in the main code, instead of saying “rm file“, I’d say “log-cmd rm file“. The log-cmd function could keep track of verboseness … and the associated debugmsg function would only produce output if a debug setting were true, too …

debugmsg() {
    if $DEBUG
    then printf -- "$*\n" >&2
    fi
}

log-cmd() {
    if $VERBOSE
    then
        printf "log-cmd executing $*\n" >&2
    fi
    if $NOEXEC
    then
        debugmsg "Execution of '$*' blocked by no-exec flag -n"
    else
        debugmsg "$($*)"
    fi
}

All that looks pretty reasonable, and indeed it works just fine with a simple invocation such as log-cmd ls -l … but when you start to want to redirect that output somewhere you’re fighting hard against bash …

log-cmd ls -l > outputfile creates outputfile OK … but it’s empty, because the redirection was interpreted by bash as being separate to the call to log-cmd. And log-cmd doesn’t return any data …

log-cmd "ls -l > output" just breaks hard, with ls complaining that there’s no file called “>” … because bash is now not helping to interpret the command-line.

At this point, stop trying to fight against bash, go and grab perl, python or perhaps ruby, and do the job properly! Because if you want this sort of feature, you probably need others … and while bash is capable of getting the job done, there are far too many mistakes to be made on the way.

Tags:

Debian and python setuptools easy_install — “error: Unexpected HTML page found”


So I’m here setting up a simple Trac environment on a Debian 5 server, and I’m about to add the essential graphviz plugin from trac-hacks.org

# easy_install http://trac-hacks.org/svn/graphvizplugin/0.11
Downloading http://trac-hacks.org/svn/graphvizplugin/0.11
error: Unexpected HTML page found at http://trac-hacks.org/svn/graphvizplugin/0.11

Wait, what? This exact same commandline works just fine on my laptop (Ubuntu, that is) … the URL is valid, I get the page I expect generated by subversion itself …

A quick Google later, it looks like the Debian packaged version can’t handle the latest subversion web pages. Thanks to coffeecoders.de we have a simple description of the problem & a workaround … upgrade easy_install underneath the Debian package.

# easy_install -U setuptools
Searching for setuptools
Reading http://pypi.python.org/simple/setuptools/
Best match: setuptools 0.6c11
Downloading http://pypi.python.org/packages/2.5/s/setuptools/setuptools-0.6c11-py2.5.egg#md5=64c94f3bf7a72a13ec83e0b24f2749b2
Processing setuptools-0.6c11-py2.5.egg
Moving setuptools-0.6c11-py2.5.egg to /usr/lib/python2.5/site-packages
Adding setuptools 0.6c11 to easy-install.pth file
Installing easy_install script to /usr/bin
Installing easy_install-2.5 script to /usr/bin

Installed /usr/lib/python2.5/site-packages/setuptools-0.6c11-py2.5.egg
Processing dependencies for setuptools
Finished processing dependencies for setuptools

This fixes the problem that easy_install has in grabbing resources from a subversion 1.5+ webserver. On the other hand it breaks the python-setuptools file …

# cd /
# md5sum -c /var/lib/dpkg/info/python-setuptools.md5sums
usr/bin/easy_install: FAILED
usr/bin/easy_install-2.5: FAILED
usr/bin/easy_install-2.4: OK
...
md5sum: WARNING: 2 of 53 computed checksums did NOT match

If you’re in the habit of verifying binaries against the package contents (e.g. with debsums), you need to know what you have just done … and look out for any upgrades of this package too (well, you could always pin this version in place, but let’s hope any future update of python-setuptools fixes this bug, eh).

Tags: , ,

A secure smarthost setup for Exim


A Linux/Unix server usually needs to be able to send email, even if only for internal reporting purposes. There’s no point in having a full-featured mail server for this, so you should configure the server to send messages via some other MTA, a “smarthost”. This could be one of your own machines, or perhaps it’s one belonging to your ISP.

Of course, when submitting to the smarthost you need to be able to differentiate yourself from spammers; this generally means you have to authenticate yourself. And if you’re doing that, it should be over an encrypted connection and you should authenticate the smarthost too. Let’s see a minimal Exim configuration that does just that …

# This is the Exim4 configuration for "leaf node".
# It provides the following behaviour :-
# • NO SMTP listeners
# • Local process submission only
# • NO local delivery
# • qualify_domain @mydomain.tld
# • /etc/aliases remapping
# • ALL mail sent to smarthost smarthost.tld
# •• with verified TLS and AUTH/LOGIN
# •• Whitelisted at smarthost

### Main
# No SMTP listeners is not a config item, it’s based on daemon invocation
# Debian packaged exim: update-exim4defaults –queuerunner queueonly

qualify_domain = mydomain.tld

### Routers
begin routers

etc_aliases:
driver = redirect
data = ${lookup{$local_part}lsearch{/etc/aliases}}

# Send everything to a smarthost
send_to_smarthost:
driver = manualroute
route_list = * smarthost.tld
transport = smarthost

### Transports
begin transports

smarthost:
driver = smtp
hosts_require_tls = smarthost.tld
hosts_require_auth = smarthost.tld
tls_verify_certificates = /etc/exim4/tls/smarthost-ca.crt

### Authenticators
begin authenticators

smarthost_login:
driver = plaintext
public_name = LOGIN
hide client_send = : MTAuserid : MTAuserpassword

Tags:

Exim tls_verify_certificates …


So you’re using Exim4, and you want to verify the TLS certificates of the MTA you are sending email to (especially useful if you want to use a smarthost).

Here’s my smarthost transport :-

smarthost:
driver = smtp
hosts_require_tls = my smarthost
tls_verify_certificates = /etc/exim4/tls/my_smarthost-ca.crt

I was reading Philip Hazel’s ‘The Exim SMTP mail server’, and it claimed that tls_verify_certificates could be a link to a directory instead of a single (possibly large) file.

However, there’s a gotcha … this is only possible if you are using the OpenSSL library with Exim4; the Debian 5 packaged version uses GnuTLS, and that can only accept the file version. You get a cryptic error in the logs if you try …

TLS error on connection to my_smarthost [IP Address] (setup_certs): Error while reading file.

See http://www.exim.org/exim-html-current/doc/html/spec_html/ch30.html for the clarifying documentation. To be fair, this has been ‘well known’ for many years …

Tags: , ,