netqmail/vpopmail Virtual Mail Hosting System Guide

1.  Introduction

Whether you're providing e-mail for just system daemons, a single server, a domain, or for many virtual domains, netqmail can easily be setup to handle your needs. This guide will help you setup netqmail for all of these scenarios with a focus on remote access and encrypted communications the whole way through.

Specifically, the packages this guide will help you with are netqmail, vpopmail, dovecot, clamav and spamassassin. These core packages will also bring in daemontools, ucspi-tcp and apache. netqmail provides the core mta functions, dovecot provides remote retrieval services, vpopmail provides virtual domain management, and spamassassin content-based spam filtering.

The last step of course is to commit yourself to the netqmail system. There are many other packages with which you could build your e-mail system. Now is the time to research and decide that netqmail is for you. We have another lovely guide centered around Postfix, or you could look into exim. It's up to you to choose the best solution for yourself; it is up to us to show you how to use netqmail.

2.  netqmail (talking to myself)

(If you want to use CRAM-MD5 authentication)
# echo mail-mta/netqmail authcram >> /etc/portage/package.use
# emerge '>=mail-mta/netqmail-1.05-r9'

Emerging netqmail will also emerge ucspi-tcp and daemontools. You can read up on ucspi-tcp and on daemontools if you like. Basically, daemontools is responsible for managing netqmail as a service while ucspi-tcp is responsible for managing the incoming TCP connections to the netqmail service.

First we have a few post-install configuration steps.

(Customize to fit your personal information)
# nano /var/qmail/control/servercert.cnf
# emerge --config netqmail

The design of netqmail has been completely around the focus of security. To this end, e-mail is never sent to the user 'root'. So now you have to select a user on your machine to receive mail that would normally be destined for 'root'. From now on in this guide, I will refer to that user as 'jdoe'.

# cd /var/qmail/alias
# echo jdoe > .qmail-root
# echo jdoe > .qmail-postmaster
# echo jdoe > .qmail-mailer-daemon

Now we want to get the netqmail delivery service up and running.

# rc-update add svscan default
# /etc/init.d/svscan start
# ln -s /var/qmail/supervise/qmail-send /service/qmail-send

We want to make sure netqmail is working correctly, so here's a quick test.

# ssh jdoe@localhost
# maildirmake .maildir
# qmail-inject root << EOF
test root e-mail!
EOF
# qmail-inject postmaster << EOF
test postmaster e-mail!
EOF
# qmail-inject jdoe << EOF
test jdoe e-mail!
EOF
(You should now have 3 e-mails in your inbox)

And that's all! Now you have a mail system that will handle mail for your local machine and the system daemons/users who utilize it.

# hostname --fqdn
example.com
# cat me
example.com
# cat defaultdomain
example.com
# cat plusdomain
example.com
# cat locals
example.com
# cat rcpthosts
example.com

# hostname --fqdn
mail.example.com
# cat me
mail.example.com
# cat defaultdomain
example.com
# cat plusdomain
example.com
# cat locals
mail.example.com
# cat rcpthosts
mail.example.com

3.  vpopmail

Although it is very popular to run vpopmail with MySQL, I do not recommend it. It is a lot easier to backup mail and user data without the uglyness of MySQL. If you use the CDB backend (the default) all data you will ever need to backup is /var/vpopmail and /var/qmail. Additionally there can arise a bunch of permission problems with MySQL if you don't pay attention. Bottom line is, using vpopmail without MySQL will save you a lot of time.

# echo net-mail/vpopmail -mysql maildrop >> /etc/portage/package.use
# emerge '>=net-mail/vpopmail-5.4.19'

At this point in time, vpopmail is ready to roll. In this guide, we will be providing virtual hosting for the domain 'example.com'. This means we need to tell vpopmail about this domain we want it to host for us. We'll also quickly add a user account for 'jdoe' while we're here.

(You only have to do this if the vadddomain step below results in "command not found")
# source /etc/profile

# vadddomain example.com postpass
(Now quickly verify the domain is setup properly)
# printf "postmaster@example.com\0postpass\0blah\0" | vchkpw `which id` 3<&0
uid=89(vpopmail) gid=89(vpopmail) groups=0(root)
(If you don't see something similar to above, then permissions somewhere are incorrect)
# vadduser jdoe@example.com jdoepass

Every domain that vpopmail creates comes with a 'postmaster' account. Here we told vpopmail that the password for the postmaster account is 'postpass'. Before vpopmail can be truly useful, we'll need to be able to receive mail via dovecot and send mail via netqmail and SMTP.

4.  Dovecot IMAP

# echo net-mail/dovecot vpopmail >> /etc/portage/package.use
# emerge '>=net-mail/dovecot-1.0.0'

# nano /etc/dovecot/dovecot.conf
(Include imap if you want to provide non-ssl connections)
protocols = imaps

(Allow access only to vpopmail users)
first_valid_uid = 89
last_valid_uid = 89
first_valid_gid = 89
last_valid_gid = 89

(Hardlinks speed things up a bit)
maildir_copy_with_hardlinks = yes
maildir_copy_preserve_filename = yes

(Get authentication data from vpopmail databases)
auth default {
  mechanisms = plain
  passdb vpopmail {
  }
  userdb vpopmail {
  }
  user = vpopmail
}

# rc-update add dovecot default
# /etc/init.d/dovecot start

Your mail client should now be able to login to the host running dovecot and retrieve mail for the virtual host. In our case, I am now able to login with the username 'jdoe@example.com' and password 'jdoepass'.

5.  netqmail (talking to the world)

Let's get SMTP up and running while making sure we don't create another spam hole for people to abuse.

# cd /var/qmail/control/
# nano conf-smtpd
(Uncomment the QMAIL_SMTP_CHECKPASSWORD variable for vchkpw)
# ln -s /var/qmail/supervise/qmail-smtpd /service/qmail-smtpd

Assuming you haven't tweaked the netqmail control files at all, netqmail will now accept mail for the example.com virtual domain and for users of the local machine. Furthermore, netqmail will relay mail for anyone who sends via 127.0.0.1 and for anyone who is able to authenticate with vpopmail. When you setup your mail client to send mail, make sure you select options like 'Server requires authentication'. In our case, I set the user as 'jdoe@example.com' and my password as 'jdoepass'. The last detail is to make sure you tell your mail client to use SSL/TLS for SMTP communication. netqmail will not let you authenticate if the session is not encrypted.

6.  SMTP Plugins (SPP)

qmail-spp adds plugin support to netqmails SMTP daemon (qmail-smtpd). Plugins are external programs which are executed after processing SMTP commands, but just before accepting it by netqmail - this lets you add extra checks on command arguments before accepting it - for instance you can check envelope recipient addresses against vpopmail or sender IP adresses against public real-time blacklist (RBL) servers.

Address Verification

Address verification is a feature that allows the server to block a sender (MAIL FROM) or recipient (RCPT TO) address until the address has been verified to be deliverable. The technique has obvious uses to reject junk mail with an unreplyable sender address.

The technique may also be useful to block mail for undeliverable recipients, for example during a dictionary or brute-force spam attack trying all possible combinations of usernames. This prevents undeliverable junk mail from entering the queue so the server does not have to waste resources trying to send MAILER-DAEMON messages back.

# nano /var/qmail/control/smtpplugins
(Uncomment the following in the [mail] section)
[mail]
plugins/mfdnscheck

(Uncomment the following in the [rcpt] section)
[rcpt]
plugins/vchkuser

qmail-smtpd will now reject every delivery attempt that has sent invalid adresses/hostnames in the MAIL FROM command and additionally checks the recipient adress against the vpopmail database.

Real Time DNS Blackhole Lists (DNSBL)

A DNSBL (aka Blocklist, aka ...) is a database of well-known spammer addresses that is queried in realtime by netqmail to proove the message is from a valid destination. Today there exist a lot of DNSBL zones for different needs or policies. Currently i have these zones in active use, as they have proven to be very reliable:

• Spamhaus ZEN
• SORBS
• SpamCop

# nano /var/qmail/control/smtpplugins
(Uncomment the following in the [rcpt] section)
[rcpt]
plugins/ifauthnext
:plugins/rblspp -r zen.spamhaus.org -r dnsbl.sorbs.net -r bl.spamcop.net

(Restart qmail-smtpd)
# svc -du /service/qmail-smtpd

7.  ClamAV and SpamAssassin

Delivery Based Content Filter

Although it is very popular to run content filters like SpamAssassin directly in the queue, this approach has a major disadvantage. Providing that you do not host spammers, it is undesirable to filter outgoing messages just for the sake of wasting resources. By the definition of the delivery based approach no outgoing message is fed through the filters, only incomming messages will be.

ClamAV

# emerge clamav

The ClamAV virus scanner should work out of the box, but you might want to tweak the location of temporary data. The default is /tmp which may be too small to extract the virus signatures if it is mounted as tmpfs.

# nano /etc/clamd.conf
(Change the following line:)
TemporaryDirectory /var/tmp

# rc-update add clamd default
# /etc/init.d/clamd start

SpamAssassin

# emerge spamassassin

Next we need to setup SpamAssassin. The configuration is easy since defaults are ok for most use cases.

# nano /etc/spamassassin/local.cf
(Since most default are ok you only need:)

# this should be low enough to catch enough spam
required_score 5.0

# we don't want the original message to be modified
report_safe 0

# rblspp does this already
skip_rbl_checks 1

# yes, we have working dns
dns_available yes

# nano /etc/conf.d/spamd
(Change SPAMD_OPTS to:)
SPAMD_OPTS="-x -m 5 -i 127.0.0.1 -A 127.0.0.1 -u vpopmail -g vpopmail"

# rc-update add spamd default
# /etc/init.d/spamd start

Maildrop

Next thing is to create a global maildrop filter configuration. During delivery qmail will export the environment variable $EXT and $HOST which can be used together with vuserinfo to figure out the correct maildir directory.

Below is a sample maildrop configuration. You should read the mailfilter manual to adapt it to your needs.

# Global maildrop filter file
SHELL="/bin/sh"

# get EXT@HOST parts from qmail
import EXT
import HOST

# uncomment if you want to enable logging
#logfile "/var/log/maildrop.log"

log "=== BEGIN maildrop processing for $EXT@$HOST ==="

# get home directory
VHOME=`/var/vpopmail/bin/vuserinfo -d $EXT@$HOST`
if ($VHOME eq "")
{
        log "VHOME ist empty, can't deliver"
        to "| /var/qmail/bin/bouncesaying 'cannot deliver to $EXT@$HOST: no such file or directory"
}

# create necessary folders
`test -d $VHOME/.maildir`
if ($RETURNCODE == 1)
{
        log "Maildir does not exist, but i will create it for you"
        `/var/qmail/bin/maildirmake $VHOME/.maildir`
}

`test -d $VHOME/.maildir/.Spam`
if ($RETURNCODE == 1)
{
        log "Spam folder does not exist, but i will create it for you"
        `/var/qmail/bin/maildirmake $VHOME/.maildir/.Spam`
        `echo Spam >> $VHOME/.maildir/subscriptions`
}

# check for virii first
xfilter "/usr/bin/clamdfilter"
if ($RETURNCODE == 1)
{
        log "=== END: VIRUS found, deleting"
        to "/dev/null"
}

# filter mail with spamassassin
xfilter "/usr/bin/spamc"
if (/^X-Spam-Flag: Yes.*$/)
{
        log "=== END: SPAM caught with spamassassin, moving to Spam folder ==="
        to "$VHOME/.maildir/.Spam/"
}

# include custom filter script
`test -r $VHOME/.mailfilter`
if ($RETURNCODE == 0)
{
        log "Including custom mailfilter from $VHOME/.mailfilter"
        exception {
                include $VHOME/.mailfilter
        }
}

log "=== END: delivery to INBOX at $VHOME ==="
to "$VHOME/.maildir/"

Since clamav does not provide a filter program, put the following in /usr/bin/clamdfilter and make it executable:

#!/bin/bash

MSG=$(cat)

echo "${MSG}" | /usr/bin/clamdscan --quiet -

if [[ $? -eq 1 ]]; then
        MSG=$(echo "${MSG}" | reformail -i "X-Virus-Status: INFECTED")
else
        MSG=$(echo "${MSG}" | reformail -i "X-Virus-Status: CLEAN")
fi

echo "${MSG}"
exit 0

8.  Extra packages

ezmlm-idx - Mailing list manager

ezmlm-idx is an add-on to ezmlm, the qmail mailing list manager. It adds multi-message threaded message retrieval from the archive, digests, message and subscription moderation, and a number of remote administration function. ezmlm-idx is also supported qmailadmin (see below).

(Just to be sure ...)
# echo net-mail/ezmlm-idx -mysql -pgsql >> /etc/portage/package.use
# emerge ezmlm-idx

ezmlm-idx will work out-of-the-box though creating new mailing lists on the command line is not simple. I recommend to use qmailadmin (see below) to create new mailing lists.

qmailadmin - Web-based administration

The popular web-based administration package for netqmail/vpopmail is qmailadmin and can be used by any mail user for account configuration. Additionally administrators (postmaster by default) can create new mail accounts, lists and much more.

# echo net-mail/qmailadmin maildrop >> /etc/portage/package.use
# emerge apache qmailadmin

(Now start apache)
# rc-update add apache2
# /etc/init.d/apache2 start

Open your favorite browser and point it to http://localhost/cgi-bin/qmailadmin.

qmHandle - qmail queue helper

If you run into problems with netqmail queues and have a hard time debugging the situation, you may want to look into qmHandle. It's a simple perl program which allows you to view and manage the netqmail message queue. Again, all you need to do is emerge net-mail/qmhandle.

9.  Final Notes

I have no final notes other than if you experience any troubles with the guide, please contact me or file a bug with Gentoo's Bugtracking Website. If you have some interesting bits you think would enhance this guide, by all means send it my way for inclusion. I love netqmail and would gladly add stuff that could possibly enhance a user's experience with the mta.

The contents of this document, unless otherwise expressly stated, are licensed under the CC-BY-SA-2.5 license. The Gentoo Name and Logo Usage Guidelines apply.