Technical

Externalizing email reception often offers many benefits: firstly, it’s [more] worry-free than servicing email internally, especially in smaller organizations where there may not be an email administrator on-call 24/7. Or, think of a situation where there is an “IT guy” who manages internal email. Then he goes on a vacation and email goes down. Now what? And even when the IT guy is present, the budget may not allow for good redundancy for email reception. What if the email server melts down. Perhaps there is a backup plan but without a stand-by server and/or perhaps virtualization option getting mail reception back online could take a day, which potentially would be a big hindrance to business.

However, outsourced email is not without its pitfalls, too. Even with a reasonably fast network connection there is more noticeable latency when accessing a remote email server as opposed to a LAN-based solution. Then there is the issue of outsourced service quality vs. the cost. Some services like Fusemail or Rackspace offer reasonable quality and fairly customizable features. But when something does go wrong you’re dependent on their response time. You’ve essentially handed away control of your email reception, for better or for worse.

Mostly, however, reception uptime is good with most well-run outsourced mail services, and the issues that more commonly crop up are related to latency and in some cases (like with Fusemail from time to time) apparent capacity issues. And if you access Fusemail with Outlook 2010′s IMAP client, you may have noticed frequent spontaneously changing message ID’s which repeatedly pop up a notification on Outlook.

The client-side issues are easy to remedy by caching inbound email on your local server. It gives you the best of both worlds: quick access to email and safety of someone monitoring mail reception 24/7 with multiple redundancies. If your local caching mail server goes down even for an extended amount of time, all you need to do is to repoint your clients to the external provider’s IMAP or POP server and you’re back in business. You may also opt to use your own outbound SMTP service (assuming you have a static IP in use) which makes it possible to isolate your domains’ SPF records to the IPs you own (as opposed to allowing anyone with an account, for example, at Fusemail to spoof mail from your domains without an SPF penalty). And if you use Fusemail, your own SMTP server will give you a peace of mind so that your outbound emails won’t trigger suspension of your account like happened to me soon after I first signed up with them (see Fusemail auto-suspends spam-suspect accounts!). Perhaps they’ve fixed that issue since then.

Setting up a caching mail service on your LAN is fairly easy with Postfix. The following tutorial assumes you already have a functioning Postfix/Dovecot setup where you’re able to send and receive email based on your requirements.

To start with, configure and test local users that you would like to correspond to outsourced email service’s mailboxes. They do not need to have the same login name, and you can also consolidate multiple external accounts to one local account. In smaller setups it’s the easiest to simply create a flat-file for user Dovecot passwords lookups.

Assuming you like to receive all email through an outsourced service (which, if you use an outsourced service, is the preferred option), you will want to restrict mail reception from the outside world only to the sending mail servers of the external mail provider of your choice. To accomplish this some restrictions are added to the local cache server’s main.cf file. The following is the configuration I use; I’ve carefully given thought of the restrictions not being too constrictive as to unnecessarily prevent connections, but on the other hand cut off connections that would not result in a successful or desired mail transit.

  | 
 copy code |
?
 
01
02
smtpd_helo_restrictions =
03
        permit_mynetworks
04
        reject_invalid_helo_hostname
05
        permit_sasl_authenticated
06
        reject_non_fqdn_helo_hostname
07
 
08
smtpd_client_restrictions =
09
        permit_mynetworks
10
        permit_sasl_authenticated
11
        check_client_access hash:$config_directory/tables/smtpd_client_access
12
        check_client_access cidr:$config_directory/tables/smtpd_client_access.cidr
13
        reject
14
 
15
smtpd_etrn_restrictions =
16
        permit_mynetworks
17
        reject
18
 
19
smtpd_sender_restrictions =
20
        reject_non_fqdn_sender
21
        reject_unknown_sender_domain
22
 
23
smtpd_recipient_restrictions =
24
        reject_non_fqdn_recipient
25
        reject_unknown_recipient_domain
26
        permit_mynetworks
27
        reject_unlisted_recipient
28
        permit_sasl_authenticated
29
        check_recipient_access hash:$config_directory/tables/smtpd_recipient_access
30
        #the following also permits mynetworks!
31
        check_recipient_access pcre:$config_directory/tables/smtpd_recipient_access.pcre
32
        reject_unauth_destination
33
 
34
smtpd_data_restrictions =
35
        reject_multi_recipient_bounce
36
        reject_unauth_pipelining
37

You will notice external hash and PCRE lookup tables “smtpd_client_access”, “smtpd_client_access.cidr”, “smtpd_recipient_access”, and “smtpd_recipient_access.pcre”. Let’s look at them next.

smtpd_client_access (hash) and smtpd_client_access.cidr (example below) list the external IP addresses you allow to connect and hence relay mail to your cache server. If the external IPs are not on this list, the connection is terminated.

Here’s an example smtpd_client_access (hash, so it’s converted to smtpd_client_access.db with postmap):

  | 
 copy code |
?
 
1
2
# some individual external server I want to allow to connect
3
100.200.100.200 PERMIT
4

And here’s an example smtpd_client_access.cidr:

  | 
 copy code |
?
 
1
2
1.2.3.4/24      OK
3
10.20.30.40      OK
4
100.200.201.0/21      OK
5

While the sending servers of an outsourced service don’t change often, they may change at any time without a warning. Thus maintaining the above list manually would be a frustrating task. To automate the process, you can cull this information from the outsourced mail service’s SPF records with a cron-scheduled shell script (note that paths relate to FreeBSD; if you run Linux, adjust them to the taste/requirements):

  | 
 copy code |
?
 
01
02
#!/bin/sh
03
 
04
ORIGINAL=/usr/local/etc/postfix/tables/smtpd_client_access.cidr
05
NEW=/tmp/postfix_clients.tmp
06
 
07
dig +short fusemail.net TXT | grep 'v=spf1' | egrep -o 'ip4:[0-9./]+' | sed 's/^ip4://' | sed 's/$/      OK/' > $NEW
08
 
09
ORIGINAL_CK=`cksum $ORIGINAL | awk '{print $1}'`
10
NEW_CK=`cksum $NEW | awk '{print $1}'`
11
 
12
if [ -s $NEW ] ; then
13
  if [ $ORIGINAL_CK != $NEW_CK ] ; then
14
    cp -f $NEW $ORIGINAL
15
    postfix reload > /dev/null 2>&1
16
  fi
17
fi
18
 
19
rm $NEW
20
 
21
exit 0
22

The above example is obviously for Fusemail, but you can modify it for other providers simply by replacing the provider domain name on the dig line.

In my configuration smtpd_recipient_access (hash) lists simply the nullroute that is often required – such as in php.ini mail configuration where you might put:

sendmail_path = /usr/sbin/sendmail -t -i -f nullroute@mydomain.com

So in smtpd_recipient_access (hash) I list:

  | 
 copy code |
?
 
1
2
nullroute@mydomain.com  PERMIT
3

Meanwhile, smtpd_recipient_access.pcre lists the users who’re allowed to receive mail externally, from the IPs you defined with smtpd_client_access/smtpd_client_access.pcre:

  | 
 copy code |
?
 
1
2
if !/^(nullroute|abuse|postmaster|user1|user2|user3)@mydomain\.com$/
3
/^/ internal
4
endif
5

Again, the above is sufficient for small configurations, but if you have dozens or more users whose external email you’re caching you may be better off storing the client_access table on a database server.

Finally, as you notice the ‘internal’ keyword being added above to user addresses that are not matched by smtpd_client_access.pcre, you want to add the following in main.cf:

  | 
 copy code |
?
 
1
2
smtpd_restriction_classes = internal, public
3
internal = permit_mynetworks, reject
4
public = permit
5

With these in place you will now receive email only from your external mail provider (and perhaps some other authorized servers if you defined one with smtpd_client_access hash table). This is important because you’re likely using your outsourced mail provider’s spam filtering, and you don’t want spammers contacting your cache mail server directly.

With the local server configured (and hopefully sufficiently tested ) you can then go ahead and create mail forwarding rules at the external provider of your choice. You would simply copy any arrived email to the corresponding email address at your local cache server. I have additionally created a rule at the external provider which prunes the mailbox after given number of time since the users will not go through and delete read email there.

You may want to allow authenticated client access to your local users so that they can access their email via IMAP or POP remotely, and perhaps over the web (like via locally installed Squirrelmail). It is also a consideration that the email at the external provider will not be synced back – if users delete email from their locally cached mailbox it will not be deleted from the mailbox at the external provider, so under normal circumstances your cached email server should be the only access point for mail. But in an event of the server melt-down it is a minor inconvenience that the external provider’s mailbox would have older emails (that perhaps were deleted locally) in it – at least the users can continue to access email while you’re getting the caching server back online!

** NOTE: I’m using the current GA/Stable version of Postfix (2.7.0). If you’re using an older version, double-check that the configuration options I propose above are available before using them! This is one reason for why I prefer to use FreeBSD for mail; the ports version of Postfix is kept well up-to-date while CentOS/RHEL Postfix package is — as you would expect from an Enterprise Linux — currently at 2.3.3. You could compile it yourself, I suppose, but I’m not up to the task since I’m not a full-time Postfix admin.