So I wrote a short script to handle the situation. When initialized at boot time through init.d or rc.d, it’ll first attempt to mount the share, but then times out in two seconds (this is a LAN NFS share so if the system offering the share is up there should not be a longer delay than that) and so the boot sequence is not slowed down terribly. (see update below) Once boot is complete, the script is run via cron every five minutes. Depending on the criticality of the share you may want to make that time shorter or longer. In this case it is a backup share which is not critical for the system’s functioning.

This technique would handle circular mounts, too, but obviously you would run into trouble if the mounts are required for successful system boot.

For this to work successfully add a marker file, such as “.myremoteservertransfers” in my example script below, in the share folder on the system exporting the share. I usually set the undeletable attribute on the file to make sure it doesn’t get accidentally deleted.

Update: Even with this code the boot sequence appears to hang until portmap times out (which takes quite a while) if the NFS share is not available at boot time. I removed the rc.d mount attempt and just shortened the cron poll period to 1 minute. That way the share will be up very quickly once it becomes available, yet the overhead caused by the periodic ping is minimal (both servers are on local LAN).

#!/bin/sh
 
SHELL=/bin/sh
PATH=/etc:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin
 
# remote system name
remotesystem=myremoteserver
 
# remote share name
remoteshare=/nfsexports/backupshare
 
# local mount point
mountpoint=/localbackups/TRANSFERS/${myremoteserver}
 
# file to indicate local mount status
testfile=$mountpoint/.myremoteservertransfers
 
# --- end variables ---
 
# ping result to the remote system (2 sec timeout); not empty is OK
remoteping=`ping -c1 -o -q -t2 ${remotesystem} | grep " 0.0%"`
 
if [ "${remoteping}" != "" ] ; then
 
   # server is available so query availability of the remote share; not empty is OK
   offsiteshare=`showmount -e ${remotesystem} | grep "${remoteshare}"`
 
   if [ "${offsiteshare}" != "" ] ; then
      if [ ! -e ${testfile} ] ; then
         mount -r -t nfs ${remotesystem}:${remoteshare} ${mountpoint}
      fi
   fi
fi
 
exit 0

However, outsourced email is not without its pitfalls, too. Even with a reasonably fast network connection there is more noticeable latency when accessing a remote email server as opposed to a LAN-based solution. Then there is the issue of outsourced service quality vs. the cost. Some services like Fusemail or Rackspace offer reasonable quality and fairly customizable features. But when something does go wrong you’re dependent on their response time. You’ve essentially handed away control of your email reception, for better or for worse.

Mostly, however, reception uptime is good with most well-run outsourced mail services, and the issues that more commonly crop up are related to latency and in some cases (like with Fusemail from time to time) apparent capacity issues. And if you access Fusemail with Outlook 2010′s IMAP client, you may have noticed frequent spontaneously changing message ID’s which repeatedly pop up a notification on Outlook.

The client-side issues are easy to remedy by caching inbound email on your local server. It gives you the best of both worlds: quick access to email and safety of someone monitoring mail reception 24/7 with multiple redundancies. If your local caching mail server goes down even for an extended amount of time, all you need to do is to repoint your clients to the external provider’s IMAP or POP server and you’re back in business. You may also opt to use your own outbound SMTP service (assuming you have a static IP in use) which makes it possible to isolate your domains’ SPF records to the IPs you own (as opposed to allowing anyone with an account, for example, at Fusemail to spoof mail from your domains without an SPF penalty). And if you use Fusemail, your own SMTP server will give you a peace of mind so that your outbound emails won’t trigger suspension of your account like happened to me soon after I first signed up with them (see Fusemail auto-suspends spam-suspect accounts!). Perhaps they’ve fixed that issue since then.

Setting up a caching mail service on your LAN is fairly easy with Postfix. The following tutorial assumes you already have a functioning Postfix/Dovecot setup where you’re able to send and receive email based on your requirements.

To start with, configure and test local users that you would like to correspond to outsourced email service’s mailboxes. They do not need to have the same login name, and you can also consolidate multiple external accounts to one local account. In smaller setups it’s the easiest to simply create a flat-file for user Dovecot passwords lookups.

Assuming you like to receive all email through an outsourced service (which, if you use an outsourced service, is the preferred option), you will want to restrict mail reception from the outside world only to the sending mail servers of the external mail provider of your choice. To accomplish this some restrictions are added to the local cache server’s main.cf file. The following is the configuration I use; I’ve carefully given thought of the restrictions not being too constrictive as to unnecessarily prevent connections, but on the other hand cut off connections that would not result in a successful or desired mail transit.

smtpd_helo_restrictions =
        permit_mynetworks
        reject_invalid_helo_hostname
        permit_sasl_authenticated
        reject_non_fqdn_helo_hostname
 
smtpd_client_restrictions =
        permit_mynetworks
        permit_sasl_authenticated
        check_client_access hash:$config_directory/tables/smtpd_client_access
        check_client_access cidr:$config_directory/tables/smtpd_client_access.cidr
        reject
 
smtpd_etrn_restrictions =
        permit_mynetworks
        reject
 
smtpd_sender_restrictions =
        reject_non_fqdn_sender
        reject_unknown_sender_domain
 
smtpd_recipient_restrictions =
        reject_non_fqdn_recipient
        reject_unknown_recipient_domain
        permit_mynetworks
        reject_unlisted_recipient
        permit_sasl_authenticated
        check_recipient_access hash:$config_directory/tables/smtpd_recipient_access
        #the following also permits mynetworks!
        check_recipient_access pcre:$config_directory/tables/smtpd_recipient_access.pcre
        reject_unauth_destination
 
smtpd_data_restrictions =
        reject_multi_recipient_bounce
        reject_unauth_pipelining

You will notice external hash and PCRE lookup tables “smtpd_client_access”, “smtpd_client_access.cidr”, “smtpd_recipient_access”, and “smtpd_recipient_access.pcre”. Let’s look at them next.

smtpd_client_access (hash) and smtpd_client_access.cidr (example below) list the external IP addresses you allow to connect and hence relay mail to your cache server. If the external IPs are not on this list, the connection is terminated.

Here’s an example smtpd_client_access (hash, so it’s converted to smtpd_client_access.db with postmap):

# some individual external server I want to allow to connect
100.200.100.200 PERMIT

And here’s an example smtpd_client_access.cidr:

1.2.3.4/24      OK
10.20.30.40      OK
100.200.201.0/21      OK

While the sending servers of an outsourced service don’t change often, they may change at any time without a warning. Thus maintaining the above list manually would be a frustrating task. To automate the process, you can cull this information from the outsourced mail service’s SPF records with a cron-scheduled shell script (note that paths relate to FreeBSD; if you run Linux, adjust them to the taste/requirements):

#!/bin/sh
 
ORIGINAL=/usr/local/etc/postfix/tables/smtpd_client_access.cidr
NEW=/tmp/postfix_clients.tmp
 
dig +short fusemail.net TXT | grep 'v=spf1' | egrep -o 'ip4:[0-9./]+' | sed 's/^ip4://' | sed 's/$/      OK/' > $NEW
 
ORIGINAL_CK=`cksum $ORIGINAL | awk '{print $1}'`
NEW_CK=`cksum $NEW | awk '{print $1}'`
 
if [ -s $NEW ] ; then
  if [ $ORIGINAL_CK != $NEW_CK ] ; then
    cp -f $NEW $ORIGINAL
    postfix reload > /dev/null 2>&1
  fi
fi
 
rm $NEW
 
exit 0

The above example is obviously for Fusemail, but you can modify it for other providers simply by replacing the provider domain name on the dig line.

In my configuration smtpd_recipient_access (hash) lists simply the nullroute that is often required – such as in php.ini mail configuration where you might put:

sendmail_path = /usr/sbin/sendmail -t -i -f nullroute@mydomain.com

So in smtpd_recipient_access (hash) I list:

nullroute@mydomain.com  PERMIT

Meanwhile, smtpd_recipient_access.pcre lists the users who’re allowed to receive mail externally, from the IPs you defined with smtpd_client_access/smtpd_client_access.pcre:

if !/^(nullroute|abuse|postmaster|user1|user2|user3)@mydomain\.com$/
/^/ internal
endif

Again, the above is sufficient for small configurations, but if you have dozens or more users whose external email you’re caching you may be better off storing the client_access table on a database server.

Finally, as you notice the ‘internal’ keyword being added above to user addresses that are not matched by smtpd_client_access.pcre, you want to add the following in main.cf:

smtpd_restriction_classes = internal, public
internal = permit_mynetworks, reject
public = permit

With these in place you will now receive email only from your external mail provider (and perhaps some other authorized servers if you defined one with smtpd_client_access hash table). This is important because you’re likely using your outsourced mail provider’s spam filtering, and you don’t want spammers contacting your cache mail server directly.

With the local server configured (and hopefully sufficiently tested ) you can then go ahead and create mail forwarding rules at the external provider of your choice. You would simply copy any arrived email to the corresponding email address at your local cache server. I have additionally created a rule at the external provider which prunes the mailbox after given number of time since the users will not go through and delete read email there.

You may want to allow authenticated client access to your local users so that they can access their email via IMAP or POP remotely, and perhaps over the web (like via locally installed Squirrelmail). It is also a consideration that the email at the external provider will not be synced back – if users delete email from their locally cached mailbox it will not be deleted from the mailbox at the external provider, so under normal circumstances your cached email server should be the only access point for mail. But in an event of the server melt-down it is a minor inconvenience that the external provider’s mailbox would have older emails (that perhaps were deleted locally) in it – at least the users can continue to access email while you’re getting the caching server back online!

** NOTE: I’m using the current GA/Stable version of Postfix (2.7.0). If you’re using an older version, double-check that the configuration options I propose above are available before using them! This is one reason for why I prefer to use FreeBSD for mail; the ports version of Postfix is kept well up-to-date while CentOS/RHEL Postfix package is — as you would expect from an Enterprise Linux — currently at 2.3.3. You could compile it yourself, I suppose, but I’m not up to the task since I’m not a full-time Postfix admin.

First (the reason for the reinstall), if there is plenty of hard drive space available, it’s good idea not to deplete it all for the sytem installations. I split a 1.3Tb RAID 5 array between the two operating systems until I realized that 1) you can’t shrink vmfs partitions and 2) by consuming all hard drive space one limits the flexibility of the system down the line. Let’s say you want to install a newer version of an operating system and decide to do a fresh install. You need space for it while you want to keep the old version around at least long enough to migrate settings and data over.

Second, while I was aware of that ESXi doesn’t offer console access beyond the “yellow and grey” terminal, I didn’t realize you have no access to the VM consoles, either. So, with CentOS or FreeBSD installed, the only way to access their consoles is via the vSphere client (someone correct me if I’m wrong — I wish I were as I’d like to have local console access to the guest OS’es).

Finally, VMware Go “doesn’t currently support ESXi servers with multiple datastores”. So if you have, say, a 3ware/LSI/AMCC RAID controller which isn’t currently supported under ESXi as a boot device but which you likely still want to use as a datastore, you’ll end up with at least two datastores. So vSphere is really the only way to go for VM management also for this reason (since LSI provides a vmware-specific driver, one may also be able to direct-connect the LSI RAID array to the VM without it being an ESXi datastore, but that’s not the configuration I’m looking for—the boot device is small and houses just ESXi while the VMs and their associated datastores are located on the array).

In the end everything’s working quite well. I like the flexibility virtualization offers.. and consolidation is useful even in a small environment (one dev machine is less than two or three dev machines ).

From time to time – usually when I’ve been getting ready to upgrade to the next major revision of FreeBSD – I’ve taken some time to research what the current pros and cons are for FreeBSD vs. some Linux distro. Always, in the end, FreeBSD has won. However, a development project I’m starting to work on will utilize Zend Server, which is only supported on handful of common Linux distros and on Windows (which is, by default, not an option as I strongly maintain that Windows is not suitable as a web server platform). There is, of course, Linux compatibility layer in FreeBSD, but as Zend doesn’t currently support it as a platform for Zend Server, I wouldn’t feel comfortable using it in a production environment.

So even though I find FreeBSD superior to Linux in many ways, I’ve now spent some time getting acquainted with Linux. I first started with Red Hat, then moved to CentOS which is the Linux distribution I’m currently testing. Now it’s not bad, per se, but I frequently come back to the thought: “Why would someone, anyone prefer THIS over a BSD system?!” The package management with yum, rpm, and the GUI overlays is easy enough, but it’s chaotic! Having to enable and disable repos, set their priorities, etc. seems unnecessarily complicated. On the FreeBSD side there is the ports collection which provides most of the software that one can imagine ever needing. The odd few items that either aren’t available in ports, or whose configuration is somehow not complete enough through ports can be easily compiled from the source tarball. Everything’s quite easy to keep track of, and to duplicate if one’s building a new system.

I’m sure some of this feeling stems from the fact that I have been using a BSD system for so long, and from the fact that I probably don’t yet know Linux well enough (say, to build the system from a scratch..). But as far as I can tell, package management is done with yum and rpm (on CentOS, say), by adjusting repository priorities, and enabling/disabling repositories. That is messy!

Well, I now have a functional development server running Zend Server with Apache, Subversion, and MySQL, and as the vendor (Zend) dictates the rules, I must continue development on Linux. Perhaps in six months time I’ll have more favorable comments about it as compared to FreeBSD… but I sort of doubt it. My guess is I’ll just learn to live with it, every now and then wistfully glancing to the direction of the BSD server.

ifconfig | grep “0xffffffff” | awk ‘{ print $2 }’ | xargs -n 1 ifconfig em0 delete

For this to work, the netmasks of the aliases and the master IP for the inteface must differ. The netmasks of the aliases are usually set to 255.255.255.255 (hence “0xffffffff”) while the netmask of the master IP is usually something different, specific to your network, e.g. 255.255.255.128 (“0xffffff80″).

Once the above command has been run, /etc/netstart can then be executed to load the remaining or reconfigured aliases (if any) from /etc/rc.conf.

Here’s my version of the script. While it bears some resemblance to Vivek’s script, it is largely rewritten. Read the script header for more information.

NOTE! In his comment James pointed out a possible bug in the script. The displayed script indeed had a problem: it was missing a backslash in front of the first dollar sign at:

eval “local fspath=\$${fsname}path”

This was caused by the script display plugin in WordPress that treated the backslash as an escape character (this has now been fixed). To be on the safe side, please download the script as a tarball. To further validate the integrity of the tarball, it should produce a md5 hash of 732ac44f11ba4484be4568e84929bb6a.

#!/bin/sh
 
# Autodump 1.5a (released 01 August 2009)
# Copyright (c) 2009 Ville Walveranta 
#
# A FreeBSD shell script to dump filesystems with full, and automatically 
# incremented incremental backups to a given directory location; this script
# was written with the intent of saving the filesystem dumps not onto a tape
# device but on another hard drive such as a different filesystem on the same 
# computer. The resulting dump files can be copied offsite with a separate 
# cron job.
#
# This script creates the necessary directory structure below the defined 
# 'BASEDIR' as well as the necessary log file. This script also ensures that
# the level 0 dump exists before creating an incremental dump; if it doesn't
# the script automatically erases the incremental files for the current week 
# (if any exist) and starts over with a level 0 dump. This way you can start 
# using the script on any day of the week and level 0 dump is automatically 
# created on the first run.
#
# When ran daily (such as from a cron job), the script creates level 0 dump
# on every Monday (beginning the ISO week), or Sunday (beginning of the U.S. 
# week) and an incremental dump on all the other days of each week. The dumps 
# are compressed with gzip and saved below the 'BASEDIR' to an automatically 
# created directory whose name is derived from the list given in 'FSNAMES'. 
# Each week's dumps are organized into subfolders with name YYYY-WW ('WW' 
# being the current week). By default three most recent weekly dumps 
# (level 0 + incrementals) are retained.
#
# The script maintains each weekly folder's date at the _beginning_ date
# of the dump (i.e. Monday or Sunday of the current week) at 00:00, not 
# at the most recent incremental's date/time.
#
# By default the root (/) and usr (/usr) filesystems are dumped. To add more  
# add a "friendly name" to the 'FSNAMES' list (it is used for the weekly folder
# names, for dump filenames, and to reference the corresponding mount point
# variable); then add the corresponding mount point variable (i.e. if you 
# add "var" to 'FSNAMES', then add a variable varpath=/var). The "path" 
# ending of the mount point variable name is required. 
#
# Since the number of incremental dumps is limited to nine (level 0 +
# incremental levels 1-9), the script will allow maximum of one dump 
# to be created per day. However, since the level incrementing is dynamic
# you can start the script on any day of the week, and run it on any
# number of days during the rest of the week and you'll always get
# level 0 plus the incremental dumps in sequential order. However, The 
# new weekly folder is always created on Monday or Sunday (as chosen by
# you). Note that the script determines whether "today's" dump exists 
# based on the modification date stamp of the most recent dump. Hence 
# it is a good idea to run this script in the early hours of each day 
# rather than in the very end of each day. Running the script, for 
# example, at 23:50 has the potential to push longer dump processes 
# over the midnight and so potentially cause the next day's dump to 
# be skipped.
#
# Written for FreeBSD 7.2 but should work on most BSD and *NIX systems with
# minor modifications.
# -------------------------------------------------------------------------
# Copyright (c) 2009 Ville Walveranta 
# <http://my.galagzee.com/2009/07/17/freebsd-dump-filesystem-shell-script>
# This script is licensed under GNU GPL version 2.0 or above, and is provided
# 'as-is' with no warranty which is to say that I'm not liable if it wipes out
# your hard drive clean or doesn't back up your precious data. However, to the 
# best or my knowledge it is working as expected -- I'm using it myself. :-)
# -------------------------------------------------------------------------
# This script was inspired by 
# FreeBSD Full / Incremental Tape Backup Shell Script
# by nixCraft project / Vivek Gite
# at <http://bash.cyberciti.biz/backup/freebsd-dump-filesystem-shell-script/>
# -------------------------------------------------------------------------
 
 
#### GLOBAL VARIABLES ###############################################
 
WEEKSTARTS=Mon      # Accepted values are "Mon" (ISO standard) or "Sun" (U.S.)
KEEPDUMPS=30        # in days; this is evaluated on the weekly level per start
                    # of the week, so '30' keeps 3-4 weekly dumps
BASEDIR=/bak/dumps
GLOBALDUMPOPTS=Lua  # add 'n' for wall notifications
LOGFILE=/var/log/dump.log
 
# to add more filesystems to be dumped add the dump name in 'FSNAMES'
# and add the corresponding mount point variable (dumpname+path=mountpoint)
FSNAMES="root usr"  # this is used for dump directory name 
                    # and to ID the path from a variable below
rootpath=/
usrpath=/usr
 
#####################################################################
 
DUMP=/sbin/dump
GZIP=/usr/bin/gzip
LOGGER=/usr/bin/logger
 
WEEKDAY=$(date +"%a")
DATE=$(date +"%Y%m%d")
HUMANDATE=$(date +"%d-%b-%Y")
HUMANDATE=`echo $HUMANDATE | tr '[:lower:]' '[:upper:]'`
HUMANTIME=$(date +"%H:%M (%Z)")
TODAYYR=$(date +"%Y")
TODAYMO=$(date +"%m")
TODAYDT=$(date +"%d")
 
# datestamp at midnight today
TODAYSTARTSTAMP=$(date -j +%s "${TODAYYR}${TODAYMO}${TODAYDT}0000")
 
# default lastdump to midnight today; it will be checked
# and and adjusted later
LASTDUMP=$TODAYSTARTSTAMP
 
# do not crete world-readable dumps!
umask 117
 
# make sure the logfile exists
if [ ! -e $LOGFILE ] ; then
   touch $LOGFILE
   chmod 660 $LOGFILE
fi
 
# make sure that entire week's incremental dumps are deposted
# in the same directory, even when a week spans new year
# NOTE: When the ending year has a partial 53rd week, there
# won't be a dump folder for the first week of the new year.
# The incremental dumps instead complete the 53rd week folder,
# even when the 1st week of the new year begins mid-week. 
# However, the dates of the incremental dump files in the 
# 53rd week folder correctly reflect the dates of the 
# beginning year.
adjust_date(){
   local dateoffset=$1
   local epochnow=$(date +%s)
   local offsetsecs=`expr $dateoffset "*" 86400`
   local newepoch=`expr $epochnow "-" $offsetsecs`
   local year=`date -r $newepoch +"%Y"`
 
   if [ "$WEEKSTARTS" = "Mon" ] ; then
      local week=`date -r $newepoch +"%W"`
   else
      local week=`date -r $newepoch +"%U"`
   fi
   NEWEPOCHISO=`date -r $newepoch +"%Y%m%d0000"`
 
   #system week starts from `0', there is no calendar week `0'
   week=`expr $week "+" 1`
   YWEEK=${year}-${week}
}
 
# determines the 'distance' from the level 0 dump in days
if [ "$WEEKSTARTS" = "Mon" ] ; then
   case $WEEKDAY in
      Mon) adjust_date 0;;
      Tue) adjust_date 1;;
      Wed) adjust_date 2;;
      Thu) adjust_date 3;;
      Fri) adjust_date 4;;
      Sat) adjust_date 5;;
      Sun) adjust_date 6;;
      *) ;;
   esac
else 
   case $WEEKDAY in
      Sun) adjust_date 0;;
      Mon) adjust_date 1;;
      Tue) adjust_date 2;;
      Wed) adjust_date 3;;
      Thu) adjust_date 4;;
      Fri) adjust_date 5;;
      Sat) adjust_date 6;;
      *) ;;
   esac
fi
 
mk_auto_dump(){
 
   local fsname=$1
 
   # get the current filesystem's path
   # as defined in the corresponding variable
   eval "local fspath=\$${fsname}path"
 
   # composite the dump path
   local dumppath=${BASEDIR}/${fsname}/${YWEEK}
 
   # make sure the dump directory for this week exists;
   # this automatically creates a new dump directory on 
   # every Monday or Sunday (as selected by 'WEEKSTARTS')
   [ ! -d $dumppath ] && mkdir -p $dumppath
 
   # get name of the last file in the current dump directory
   local lastfile=`ls -ltr $dumppath | grep -v "^d" | tail -n 1 | awk '{ print $9 }'`
 
   # assume that the 'lastfile', if it exists, was not created today
   local dumped_today=false
 
   # if a file exists, check its modification date; 
   # if it is at or after 00:00 today, set a flag to skip the dump
   if [ "$lastfile" != "" ] ; then
      local fq_lastfile=${dumppath}/$lastfile
      if [ -e $fq_lastfile ] ; then
         # get the last modification time for the most recently created dumpfile
         LASTDUMP=`stat -f %m $fq_lastfile`
         if [ $LASTDUMP -ge $TODAYSTARTSTAMP ] ; then
            local dumped_today=true
         fi
      fi
 
      # get the first and the last dump level for this directory
      local levelcommand="ls $dumppath | sed -e 's/^[[:digit:]]*\_//' | sed -e 's/\..*$//'"
      local firstlevel=`eval $levelcommand | head -n 1`
      local lastlevel=`eval $levelcommand | tail -n 1`
 
      # make sure level zero dump exists;
      # if it doesn't, start over
      if [ "$firstlevel" != "0" ] ; then
         # it doesn't matter if a previous dump exists from today
         # since we're starting over as level 0 dump is missing
         local dumped_today=false
         local dumplevel=0
         rm -f $dumppath/*.gz
      else
         # otherwise just increment the dump level
         # for levels 1-6, i.e. normally Tuesday thru Sunday
         local dumplevel=`expr $lastlevel "+" 1`
      fi
   else
      # no dump exists in this week's folder; reset level to '0'
      local dumplevel=0
   fi
 
   # skip the entire dump process if a dumpfile has
   # already been created for this filesystem today
   if [ "$dumped_today" = "false" ] ; then  
 
      # define the dump filename
      local dumpfn=${DATE}_${dumplevel}
 
      echo ---------------- >> $LOGFILE
      echo >> $LOGFILE
      echo BEGINNING LEVEL $dumplevel DUMP OF \'$fsname\' \(${fspath}\) FILESYSTEM ON $HUMANDATE AT $HUMANTIME >> $LOGFILE
      echo >> $LOGFILE
      echo Creating a snapshot of \'$fspath\'.. >> $LOGFILE
      # execute the dump
      $DUMP -$dumplevel -$GLOBALDUMPOPTS -f ${dumppath}/${dumpfn} $fspath >> $LOGFILE 2>&1
      local dumpresult=$?
 
      if [ "$dumpresult" != "0" ] ; then
         # log the dump result to syslog
         $LOGGER "$DUMP LEVEL $dumplevel DUMP OF $fsname (${fspath}) FAILED!"
 
         echo "*** DUMP FAILED - LEVEL $dumplevel DUMP of $fsname (${fspath}) ***" >> $LOGFILE
         echo >> $LOGFILE
      else
         # log the dump result to syslog
         $LOGGER "LEVEL $dumplevel DUMP of $fsname (${fspath}) COMPLETED SUCCESSFULLY!"
 
         echo >> $LOGFILE
         # compress the dump
         echo Compressing the dumpfile \'${dumpfn}\'.. >> $LOGFILE
         $GZIP -v ${dumppath}/${dumpfn} >> $LOGFILE 2>&1
         echo DONE >> $LOGFILE
         echo >> $LOGFILE
 
         # make sure dumps are not world readable (security risk!)
         echo Updating dumpfile \'${dumpfn}.gz\' permissions.. >> $LOGFILE
         chmod -v -v 440 ${dumppath}/${dumpfn}.gz >> $LOGFILE 2>&1
         echo DONE >> $LOGFILE
         echo >> $LOGFILE
 
         # reset current dump dir's timestamp to that of the level 0 dump
         touch -t ${NEWEPOCHISO} ${dumppath}
 
         # delete old dumps
         echo Deleting old \'$fsname\' dumpfiles.. >> $LOGFILE
         find $BASEDIR/$fsname -mtime +$KEEPDUMPS -maxdepth 1 -print -exec rm -rf {} \; >> $LOGFILE 2>&1
         echo DONE >> $LOGFILE
         echo >> $LOGFILE
      fi
   else
      local lastdump_readable=`date -j -r $LASTDUMP +"%H:%M"`
      local lastdump_readableZ=`date -j -r $LASTDUMP +"%Z"`
      local lastdumpmsg="Autodump for filesystem '$fsname' ($fspath) has already been executed today at $lastdump_readable ($lastdump_readableZ)."
      echo $lastdumpmsg
      $LOGGER $lastdumpmsg
   fi
}
 
 
# Dump filesystems defined in 'FSNAMES'
#
# Monday or Sunday (as selected by 'WEEKSTARTS') starts with 
# the level 0 dump, with incrementals created through the rest of 
# the week (autoincremented). If the level 0 dump is missing in 
# the current week's folder for filesystem currently being backed 
# up, it is created automatically instead of an incremental dump, 
# no matter what day of the week it is.
for f in $FSNAMES
do
   mk_auto_dump $f
done

Here’re couple of useful sites for those wondering which OS to choose:

Polishlinux.org – Compare distros: FreeBSD vs. Debian – Comparison data is up to date and there are a lot of good user comments to sift through. You can also choose other distros to compare to.

Wikipedia – Comparison of BSD operating systems

And lastly, a good example of why the sheer number of Linux distros is disorienting: DistroWatch lists at least a few hundred Linux distros (plus couple of BSD derivatives).

Meanwhile, my employer’s email has been running on Exchange for several years, starting preceding my time with the company.  It has been a grief, though I’m sure it’s partially due to the fact that the the mail server is also the domain controller of a small office LAN. But why should it be? Qmail or Postfix run quite well on a Linux/*BSD server with Apache, MySQL, BIND. So I’ve been looking forward getting rid of Exchange, and migrating to Postfix/Dovecot system until, again, I started thinking that perhaps it’s not worth the stress to run an internal mail server. I’m the only person tending to it and, say, if I’m on a vacation and the mail goes down, it would not be good.

Once I started considering outsourcing email an option, I started evaluating various services. Fusemail and Mailtrust quickly bubbled to the top. Fusemail has more features, and the deciding factors (in Fusemail’s favor) were the ability to adjust the spam filtering (Mailtrust only has “on” or “off” options which is a bit scary — if the filtering is too stringent or too lenient, there’d be nothing that could be done about it.. Mailtrust’s rep suggested that I might want to look into an external spam filtering solution if I wanted more control.. but no thanks; I had been running Katharion for mail filtering for several months which worked ok, but if I was going to outsource the mail, I wanted an integrated solution), and the ability to increase a mailbox allocation for an individual user by purchasing more user accounts and allocating their mailbox allowance to the existing users. Mailtrust is fixed to 10Gb.

On the web there is about 50/50 comments for and against the quality of support for both Fusemail and Mailtrust, so from the comments alone it was impossible to deduce which service would have better support. Pre-sales support was slightly better on Fusemail side, and the few quirks ran across during setup have been addressed satisfactorily.

Strike One

Tonight (Saturday evening) around 18:00 my user account under my employer’s master account suddenly disappeared.  I access mail from Outlook via IMAP, and suddenly Outlook prompted for the account password. So I logged in to Fusemail admin account and clicked on my user name. [Paraphrasing] “Cannot edit terminated user account”.  What?! To terminate a user account in Fusemail you have to check the checkbox next to the user name, click “Terminate”, check another checkbox (“yes, I’m sure I want to do that”), and then click on “Yes”. Only then does a user account get removed, or scheduled for deletion as it takes many, many hours for the username actually be purged from the system so that it can be taken into use again. I most certainly did not execute those steps.  I’m the only one with access to that admin account, and the password is sufficiently complex so that it’s very unlikely the account would’ve been compromised. This leaves system error as the most likely cause.  I called the emergency support around 18:30 and left a message (they claim to have someone on call), then again again around 20:00, and also opened an “Urgent” support ticket through their support system at 22:40.  It’s now over six hours since my first “emergency” support request, so I can only assume the on-call person has gone to party (or that they don’t have an on-call tech in the first place). The emergency support number instructs the caller that “while the support technician is not immediately available, it does not mean that support would not be available immediately”. It’s looking like they were wrong.

I didn’t lose a tremendous amount of email (and perhaps Fusemail can restore it), but during this downtime emails to my account which has multiple “admin” aliases are being rejected.  If I was running my own mail server I could obviously have fixed a problem already, but an outsourced solution is supposed to *reduce* system management stress.

Longevity of this outsourcing attempt depends largely on how Fusemail will deal with this situation. Having to reconfigure my user account and its associated aliases would be annoying, but more than restore I want to know what caused the problem, can they be sure to prevent it from recurring, and what’s the deal with the non-existent emergency support.

If the deleted account would’ve been that of the CEO of my employer, or my personal primary account (which I have also outsourced to Fusemail in a separate account), this first strike would’ve likely been also the last for Fusemail.

–

Couple of considerations for those who’re comparing, say, Fusemail and Mailtrust, or considering mail outsourcing in the first place:

• Forward/distribution management is currently better implemented in Mailtrust.  It’s workable in Fusemail, but it’s more straightforward in Mailtrust. If this is an important feature to you, pay attention when you’re comparing the services.
• Secure connections (SMTP, IMAP, POP) work better with Mailtrust than with Fusemail. Fusemail is supposedly looking into this. Not a huge issue for me since the SMTP traffic is generally not encrypted anyway, so encrypting the last leg (from the service to the client) isn’t very significant.
• Fusemail’s IMAP is not blazingly fast even when accessed from a fast net connection. Same goes occasionally for their web client. They are, however, generally within acceptable limits.
• A general comment if you’re using SPF: when you use a service provider’s SMTP servers you can’t positively lock down who’s authorized to send mail for your domain. If someone who’s hosting their mail at Fusemail decides to send spam spoofing one of my domains, they’ll appear as authorized for the recipient’s spam filter since I’ve authorized Fusemail’s SMTP servers in my domains’ SPF records.
• Test (!) the support of different providers by sending them a support request on Saturday evening. See when you get a response. Fusemail claims on their website: “24x7x365 Support”, but I’m now finding that it is not completely solid; it should instead read: “You can leave us a message 24x7x365″.
• If you don’t have large number of users to support and they use IMAP to access the remote email, consider setting up backup mailboxes at gmail (free!), and creating a mail rule (available at least in Fusemail) which automatically copies those backup mailboxes for all inbound email.

1. Create a perimeter file, like so:
   touch -t yyyymmddHHMM marker_date

2. List files older than the marker_date:
   find . -type f ! -newer marker_date -ls
   Of course, instead of `-ls’ parameter (to list), you can use `-print’ and a pipe to xargs to, for example, delete the selected files, etc.

Likewise, for a range of dates:

1. Create the perimeter files:
   touch -t yyyymmddHHMM range_start
   touch -t yyyymmddHHMM range_end

2. List the files between the range dates:
   find . -type f -newer range_start ! -newer range_end -ls