Whitepaper Home

Build a Mail Router with Qmail
Copyright © 2008 Rus Shuler, Enterprise Architect

What Does It Do?

It routes email. Specifically, it re-writes the email delivery address in the mail envelope, then delivers the email(s).

It could be used to-

 

What Do You Need?

Basically, you need a unix server running a patched and modified version of qmail, plus some custom scripts supplied here.

Specifically-

 

Assumptions

This is not a qmail tutorial nor a one-click install solution.

It assumes-

 

Setup Qmail + Add-Ons

Install qmail, patch it to version 1.05+, configure it, get it running, and add some things to it.

 

Setup Scripts

/etc/tcp.smtp

This script configures tcpserver, which is in the ucspi-tcp package. This dictates what can connect and what handles the connection.

192.168.0.:allow,RELAYCLIENT="",QMAILQUEUE="/var/qmail/filters/routemail.filter"
127.:allow,RELAYCLIENT="",QMAILQUEUE="/var/qmail/filters/routemail.filter"
:allow,QMAILQUEUE="/var/qmail/filters/routemail.filter"

After you edit this file you must convert it to a binary file (/etc/tcp.smtp.cdb) for tcpserver. Use a script such as-

#!/bin/bash
tcprules /etc/tcp.smtp.cdb /etc/tcp.smtp.tmp < /etc/tcp.smtp

/var/qmail/filters/routemail.filter

This is a shell script which runs qmail-qfilter. Qmail-qfilter invokes our Perl script which does the address transformation.

#!/bin/sh
# invoked from /etc/tcp.smtp, runs qmail-qfilter instead of qmail-queue
# qmail-qfilter invokes routemail.filter.pl to transform recipient addresses
exec /var/qmail/bin/qmail-qfilter /var/qmail/filters/routemail.filter.pl

/var/qmail/filters/routemail.filter.pl

This Perl script reads a file line by line and transforms the email envelope addresses. The address transformations are stored in /var/qmail/addresses.

#!/usr/bin/perl
#
# rus.shuler - routemail.filter.pl - nov.2006
#
# - reads /var/qmail/addresses for email addresses to transform.
# - transforms mail envelope recipient(s) for redelivery by qmail by reading
#   file descriptor 3 and writing to descriptor 4.
# - reads mail msgs from stdin and writes to stdout.
# - this script is invoked via /etc/tcp.smtp and /var/qmail/filters/routemail.filter.
# - qmail-qfilter is an add-on for qmail that makes this possible, see
#   http://untroubled.org/qmail-qfilter/

# read the address file and put the addresses in an array
@addresses = ();
$addrIndex = 0;
open(ADDRESSES, " ) {
    if ( $line =~ m/^\s*(.*)\s*,\s*(.*)\s*\b/ && $line !~ m/.*#/ ) {
        $addresses[ $addrIndex++ ] = $1 . "," . $2;
    }
}
close ADDRESSES;

# get envelope sender and envelope recipient from file descriptor 3, format is-
# Fsender@somewhere.com\0Trecipient@here.com\0\0
# \0 = ascii zero (null), we translate these to tildes to make it easier to work with

# read the envelope from file descriptor 3
open(ENVIN, "<&=3") or die "Can't open file descriptor 3 ($!), stopped";
$num_nulls = -1;
if ( $line =  ) {
    $num_nulls = $line =~ tr/\0/~/;
    `logger -t routemail.filter.pl -p mail.info Envelope from qmail-smtpd is [$line]`;
    @envelope = split("~", $line);
} else {
    `logger -t routemail.filter.pl -p mail.error No envelope from qmail-smtpd, rejecting`;
    exit 31;
}
close ENVIN;

# iterate over the envelope addresses, remember the sender and transform the recipient(s)
$sender = "";
for ( $i = 0 ; $i < scalar(@envelope) ; $i++ ) {

    # differentiate between the sender and recipient addresses
    if ( $envelope[$i] =~ m/^F(.+)$/ ) {
        
        # remember the sender address
        $sender = $1;
        
    } elsif ( $envelope[$i] =~ m/^T/ ) {
        
        # transform recipient addresses
        for ( $addrIndex = 0 ; $addrIndex < scalar(@addresses) ; $addrIndex++ ) {
            ( $oldAddr, $newAddr ) = split(/,/, $addresses[ $addrIndex ]);
            $envelope[$i] =~ s/$oldAddr/$newAddr/ig;
        }
        
    }

}

# rebuild the envelope and write it to file descriptor 4
$line = "F$sender~";
foreach $address ( @envelope ) {
    if ( $address =~ m/^T/ ) {
        $line = "$line$address~";
    }
}
$line = "$line~";
`logger -t routemail.filter.pl -p mail.info Envelope to qmail-queue is [$line]`;
$line =~ tr/~/\0/;
open(ENVOUT, ">&=4") or die "Can't open file descriptor 4 ($!), stopped";
print ENVOUT $line;
close ENVOUT;

# open stdin to read the email message from qmail-smtpd via qmail-qfilter
open(MSGIN, "<-") or die "Can't open STDIN ($!), stopped";

# open stdout to write the message to
open(MSGOUT, ">-") or die "Can't open STDOUT ($!), stopped";

# define a var to track when the headers are done
# ie $header=0 means we're done reading the headers
$header = 1;

# we're going to look for the Sender: header
$senderHdr = "";

# read email line by line and write to stdout
while ( $line =  ) {

    # do header processing    
    if ( $header ) {
        # look for and capture the Sender: header address
        if ( $line =~ m/^[Ss]ender:\s*(.+)\s*\n/ ) { $senderHdr = $1; }
    }
    
    # detect the end of the headers, a line with one char (a newline) should do it
    # see http://cr.yp.to/immhf/header.html for more info
    if ( $header == 1 && length($line) == 1 ) {
        
        $header = 0;
        
        # if there isn't a Sender: header then create one with the envelope sender
        if ( length($senderHdr) == 0 ) {
            print MSGOUT "Sender: " . $sender . "\n";
        }
        
    }
    
    print MSGOUT $line;

}   # next email line

close MSGIN;
close MSGOUT;

# exit codes for qmail-qfilter are:
#
#   0  = accept message and deliver it
#   99 = accept message but do not deliver it
#   31 = reject message with a 554

exit 0;

# end

Permissions!

routemail.filter and routemail.filter.pl must be owned by root with their group set to qmail. They must be executable by everyone.

This will set everything correctly-

cd /var/qmail/filters
chown root:qmail routemail.filter*; chmod 755 routemail.filter*

A directory listing of /var/qmail/filters should show this-

-rwxr-xr-x   1 root qmail  231 Nov 28 09:57 routemail.filter
-rwxr-xr-x   1 root qmail 5030 Dec  1 11:25 routemail.filter.pl

Permissions are very important in qmail. If something isn't working as expected, check your permissions.

 

Configuring Transformations in /var/qmail/addresses

Here is an example /var/qmail/addresses file-

# addresses
# ---------
#
# read by /var/qmail/filters/routemail.filter.pl
#
# format is [email address to replace], [replacement email address], or on more
# general terms [anything to replace], [replacement]
#
# AN ADDRESS MAY GO THROUGH MULTIPLE TRANSFORMATIONS!
# why?  because the entire file is read for each address and processed line by line
# e.g. support@mail.yourcompanysolutions.com > support@yourcompany.com > someone@yourcompanysolutions.com

# ---------------------
# local transformations
# ---------------------

# make sure all local accounts are accounted for if they may get mail we care about.

postmaster@mail.mailserver.com,     support@yourcompany.com

# -----------------------
# corporate boxes & misc.
# -----------------------

# use a "~T" to add another recipient address in the mail envelope
support@yourcompany.com,            dave@yourcompany.com~Ted@yourcompany.com

# ------------
# ex-employees
# ------------

hannibal@yourcompany.com,           adam@yourcompany.com

# -------------------------------------
# outbound or temporary transformations
# -------------------------------------

contractor@yourcompany.com,         judy@carolina.rr.com

# --------------
# bullhorn users
# --------------

# all bullhorn users must be specified

jeff@yourcompany.com,               jeff.logan@mail.bullhorn.com
mark@yourcompany.com,               mark.adams@mail.bullhorn.com

# this should be the last transform, which will send all yourcompany.com mail to
# another server.  this must be the last transformation.

@yourcompany.com,                   @yourcompany.provider.com

 

Other Stuff

Server With Dynamic IP

If you run this on a server with a dynamic IP address, e.g. at home, many mail servers will not accept your email. Yahoo is a good example. Yahoo will either not accept the email or relegate it to the spam folder. You can get around this by relaying mail through your ISP's mail server.

Edit /var/qmail/control/smtproutes-

# Artificial SMTP routes.
# Each route has the form [domain]:[relay], without any extra spaces.  If domain
# matches host (cmd line param), qmail-remote will connect to relay, as if host 
# had relay as its only MX.  (It will also avoid doing any CNAME lookups on recip.)
# host may include a colon and a port number to use instead of the normal SMTP port.
#
# eg: somedomain.com:1.2.3.4
#     somedomain.com:mail.somedomain.com
#
# make sure we deliver mail to [user]@theshulers.com to ourselves.  if this line
# weren't here then qmail-remote would attempt to connect to the firewall's
# external address which doesn't work from inside.  
# this takes care of bounces generated from MAILER-DAEMON (qmail).
theshulers.com:127.0.0.1

# route all mail through our local provider's server
:smtp-server.carolina.rr.com

More on qmail-qfilter

Here are some notes regarding qmail-qfilter.

qmail-qfilter(1)                                                qmail-qfilter(1)

NAME
       qmail-qfilter - front end for qmail-queue that does filtering

SYNOPSIS
       qmail-qfilter filter [ -- filter ...  ]

DESCRIPTION
       qmail-qfilter  sends  the  message text through each of the filter 
       commands named on the command line.  Each filter is run separately, with 
       standard input opened to the input email, and standard output opened to a 
       new temporary  file  that  will become the input to either the next 
       filter, or qmail-queue.  If the filter does not modify the message it 
       passes unchanged to the next step.  It also makes the envelope available 
       to each filter as file descriptor 3.  File descriptor 4 is  opened to a 
       new temporary file for the modified envelope, allowing the filter to 
       modify the envelope or the message.  If the fil- ter does not modify the 
       envelope, the envelope remains unchanged for either the next filter or 
       qmail-queue.  This provides compatibility  for existing filters that do 
       not know about the envelope.  qmail-qfilter also opens up file descriptor 
       5 to a temporary file.  If this file is empty after all the filters have 
       executed, its contents are read and used to specify  a program to execute 
       in place of qmail-queue.  Each filter on the command line in separated 
       with --.

RETURN VALUES
       Returns  51  (out of memory), 53 (write error), or 81 (internal error) if 
       it canít create the temporary files or has prob- lems executing the 
       filters.  Returns 91 (bad envelope data) if it canít read or parse the 
       envelope  data.   If  a  filter returns anything other than 0 or 99, 
       qmail-qfilter returns its exit code.  If a filter returns 99, qmail-
       qfilter returns 0 immediately without running any other filters. 
       Otherwise returns the exit code of qmail-queue.

ENVIRONMENT
       For compatibility with previous versions, qmail-qfilter sets QMAILUSER and 
       QMAILHOST to the user and host portions  of  the envelope  sender 
       address, and unsets QMAILNAME.  It also sets QMAILRCPTS to the list of 
       envelope recipients, each followed by a newline.

       It also sets ENVSIZE to the size of the envelope, MSGSIZE to the length 
       of the message, and  NUMRCPTS  to  the  number  of recipients. These 
       values are updated before each filter is run.

       If QQF_QMAILQUEUE is set, its value is used in place of qmail-queue.

SEE ALSO
       qmail-queue(8)

NOTES
       $QMAILQUEUE  is  deliberately  not used to override qmail-queue in order 
       to avoid recursive loops with configurations that set $QMAILQUEUE to 
       invoke qmail-qfilter itself.

WARNINGS
       If you are using qmail-inject -n as one of the filters, you may want to 
       unset MAILUSER, USER, and LOGNAME by using env  -u QMAILNAME  -u MAILNAME 
       -u NAME qmail-inject -n as the command to invoke qmail-inject.  Note that 
       some the env command with some OSís doesnít support the -u option.

       A message with an excessive number of recipients (more than 64K bytes of 
       recipient data on Linux) will cause execution  of the filter programs to 
       fail, and for the message to be rejected.

       The  same  temporary  file is reused for file descriptor 5 for each 
       filter.  Make sure each filter writes a trailing ASCII NUL byte following 
       the program name, as multiple filters could otherwise overwrite the value 
       in undesirable ways.

                                                                qmail-qfilter(1)

--

The README file


qmail-qfilter
qmail-queue multi-filter front end
Bruce Guenter 
Version 2.1
2005-08-12

This program allows the body and/or envelope of a message to be filtered
through a series of filters before being passed to the real qmail-queue
program, and injected into the qmail queue.

A mailing list has been set up to discuss this and other packages.
To subscribe, send an email to:
	bgware-subscribe@lists.untroubled.org
A mailing list archive is available at:
	http://lists.untroubled.org/?list=bgware

Development versions of qmail-qfilter are available via Subversion at:
	svn://bruce-guenter.dyndns.org/qmail-qfilter/trunk

Requirements:

- bglibs is required for system dependancies.

- This program is designed to take advantage of my QMAILQUEUE patch,
  which causes programs that would execute qmail-queue (such as
  qmail-smtpd etc.) to execute an alternative program.

How to install:

- Check the definitions at the top of qmail-qfilter.c, especially the
  value of TMPDIR.  This should be set to a temporary directory that
  only the executor of qmail-qfilter has write access to.
- Check the conf-* files for appropriate values for your compiler and
  linker, and installation paths.
- Run "make"
- As root, run "make install"

How to use, with the QMAILQUEUE patch applied to qmail:

- Create a script containing an invocation of qmail-qfilter.  For
  example, a script that uses qmail-inject as a front end to qmail-queue
  would contain:
	#!/bin/sh
	exec /path/to/qmail-qfilter /var/qmail/bin/qmail-inject -n
- Set the environment variable QMAILQUEUE to the location of this
  script.  For example, in a SMTP rules files, put:
	A.B.C.D:allow,RELAYCLIENT="",QMAILQUEUE="/usr/local/bin/qmail-qftest"
  and rebuild the SMTP CDB file.
- You're all set!  In our example, all messages sent from the IP A.B.C.D
  will have their content filtered through qmail-inject, which will add
  missing "From:", "Date:", and "Message-Id:" headers.

How to use, without the QMAILQUEUE patch:
- Change the definition of QMAIL_QUEUE in qmail-qfilter.c to a different
  value, either by editing the source file or by modifying the DEFINES
  line in the Makefile to read:
	-DQMAIL_QUEUE=\"/var/qmail/bin/qmail-queue-old\"
- Compile qmail-qfilter.
- Rename qmail-queue to the new filename specified above.
- Create a script to replace qmail-queue that contains an invocation of
  qmail-qfilter, as described in the previous example.
- You're all set!  All mail entering the queue will be filtered by your
  filter.

Notes on writing a filter program:
- If you want to block an email, exit from the filter with code 31.
  This will cause qmail-qfilter to exit with the same error code, and
  qmail-smtpd (for example) to send an error code to the client.
- If you want to silently drop an email, exit with code 99.
- The filter script that executes qmail-queue MUST NOT be setuid, and
  MUST BE readable.  Only the real qmail-queue binary needs to be
  setuid.

See the scripts in the "samples" directory for example scripts.

This program is Copyright(C) 2001,2004-2005 Bruce Guenter, and may be
copied according to the GNU GENERAL PUBLIC LICENSE (GPL) Version 2 or a
later version.  A copy of this license is included with this package.
This package comes with no warranty of any kind.


--

From http://untroubled.org/qmail-qfilter/


qmail-qfilter
qmail-queue multi-filter front end
Bruce Guenter 
Version 2.1
2005-08-12

This program allows the body and/or envelope of a message to be filtered
through a series of filters before being passed to the real qmail-queue
program, and injected into the qmail queue.

A mailing list has been set up to discuss this and other packages.
To subscribe, send an email to:
	bgware-subscribe@lists.untroubled.org
A mailing list archive is available at:
	http://lists.untroubled.org/?list=bgware

Development versions of qmail-qfilter are available via Subversion at:
	svn://bruce-guenter.dyndns.org/qmail-qfilter/trunk

Requirements:

- bglibs is required for system dependancies.

- This program is designed to take advantage of my QMAILQUEUE patch,
  which causes programs that would execute qmail-queue (such as
  qmail-smtpd etc.) to execute an alternative program.

How to install:

- Check the definitions at the top of qmail-qfilter.c, especially the
  value of TMPDIR.  This should be set to a temporary directory that
  only the executor of qmail-qfilter has write access to.
- Check the conf-* files for appropriate values for your compiler and
  linker, and installation paths.
- Run "make"
- As root, run "make install"

How to use, with the QMAILQUEUE patch applied to qmail:

- Create a script containing an invocation of qmail-qfilter.  For
  example, a script that uses qmail-inject as a front end to qmail-queue
  would contain:
	#!/bin/sh
	exec /path/to/qmail-qfilter /var/qmail/bin/qmail-inject -n
- Set the environment variable QMAILQUEUE to the location of this
  script.  For example, in a SMTP rules files, put:
	A.B.C.D:allow,RELAYCLIENT="",QMAILQUEUE="/usr/local/bin/qmail-qftest"
  and rebuild the SMTP CDB file.
- You're all set!  In our example, all messages sent from the IP A.B.C.D
  will have their content filtered through qmail-inject, which will add
  missing "From:", "Date:", and "Message-Id:" headers.

How to use, without the QMAILQUEUE patch:
- Change the definition of QMAIL_QUEUE in qmail-qfilter.c to a different
  value, either by editing the source file or by modifying the DEFINES
  line in the Makefile to read:
	-DQMAIL_QUEUE=\"/var/qmail/bin/qmail-queue-old\"
- Compile qmail-qfilter.
- Rename qmail-queue to the new filename specified above.
- Create a script to replace qmail-queue that contains an invocation of
  qmail-qfilter, as described in the previous example.
- You're all set!  All mail entering the queue will be filtered by your
  filter.

Notes on writing a filter program:
- If you want to block an email, exit from the filter with code 31.
  This will cause qmail-qfilter to exit with the same error code, and
  qmail-smtpd (for example) to send an error code to the client.
- If you want to silently drop an email, exit with code 99.
- The filter script that executes qmail-queue MUST NOT be setuid, and
  MUST BE readable.  Only the real qmail-queue binary needs to be
  setuid.

 

Enjoy!

Hope I haven't forgotten anything important. This really does assume you already have qmail running and want to add routing functionality. Send comments, suggestions and corrections to russhuler@yahoo.com.



All company names, trademarks, and products are the property of their respective owners.
Correspondence regarding this whitepaper should be directed to russhuler@yahoo.com.