$ aptitude install fail2ban

Then I simply create a local configuration file where I’ll put all my custom config:

$ touch /etc/fail2ban/jail.local

Here is the content of that file:

[DEFAULT]
# Do not filter connexion from my apartment and from the server itself
ignoreip  = 127.0.0.1 88.123.123.123 91.123.123.123
# Ban for a week
bantime   = 604800
maxretry  = 3
destemail = kevin@deldycke.com
banaction = iptables-allports
action    = %(action_mwl)s

[ssh]
enabled  = true
port     = 22
maxretry = 2

[ssh-ddos]
enabled = true
port     = 22

[apache]
# Apache basic auth
enabled   = true
maxretry  = 3
# Ban for 1 hour
bantime   = 3600

[apache-noscript]
enabled = true

[apache-overflows]
enabled = true

[apache-badbots]
enabled  = true
filter   = apache-badbots
port     = http,https
action   = iptables-allports
logpath  = /var/log/apache*/*access.log
maxretry = 1

[apache-nohome]
enabled  = true
filter   = apache-nohome
port     = http,https
action   = iptables-allports
logpath  = /var/log/apache*/*access.log
maxretry = 1

[exim]
enabled  = true
filter   = exim
port     = smtp,ssmtp
action   = iptables-allports
logpath  = /var/log/exim*/rejectlog
maxretry = 1

[exim-relay]
enabled  = true
filter   = exim-relay
port     = smtp,ssmtp
action   = iptables-allports
logpath  = /var/log/exim*/rejectlog
maxretry = 1

While adjusting Fail2Ban, I was surprised by how sensitive this software is. It can just refuse to start without any notice in the log or on the command line. Even if its log_level variable is set to 4 (= DEBUG) in /etc/fail2ban/fail2ban.conf.

In such a case, a sure way to find the culprit is to use a brute force debugging method: first set all the enabled variable of your jail.local‘s sections to false. Then activate one section after another until Fail2Ban refuse to restart.

For me, the problem was that I forgot to add my custom exim-relay filter to Fail2Ban. So I fixed my issue by creating an empty file at /etc/fail2ban/filter.d/exim-relay.conf in which I pasted the following content:

# Based on default exim.conf filter by Cyril Jaquier
# Real life exemaple:
# 2009-07-02 08:16:42 H=118-167-129-21.dynamic.hinet.net (91.121.198.84) [118.167.129.21] F=<titieueue@hotmail.com> rejected RCPT <s2288@mail2000.com.tw>: relay not permitted

[Definition]

# Option:  failregex
# Notes.:  regex to match use of my exim mail server as a relay it does not
#          allow.
# Values:  TEXT
#
failregex = \[<HOST>\] .*(?:relay not permitted)

# Option:  ignoreregex
# Notes.:  regex to ignore. If this regex matches, the line is ignored.
# Values:  TEXT
#
ignoreregex =

Speaking of custom filters, here is one to filter DFind scans (file located at /etc/fail2ban/filter.d/apache-w00tw00t.conf):

# Based on http://howflow.com/tricks/block_w00tw00t_scan_hosts_with_fail2ban
# Real life exemaple:
# [Sat Jun 27 16:43:08 2009] [error] [client 94.23.57.77] client sent HTTP/1.1 request without hostname (see RFC2616 section 14.23): /w00tw00t.at.ISC.SANS.DFind:)

[Definition]

# Option:  failregex
# Notes.:  regex to match the w00tw00t scan messages in the logfile.
# Values:  TEXT
failregex = ^.*\[client <HOST>\].*w00tw00t\.at\.ISC\.SANS\.DFind.*

# Option:  ignoreregex
# Notes.:  regex to ignore. If this regex matches, the line is ignored.
# Values:  TEXT
ignoreregex =

And here is the corresponding section from my jail.local file:

[apache-w00tw00t]
enabled  = true
filter   = apache-w00tw00t
action   = iptables-allports
logpath  = /var/log/apache*/*error.log
maxretry = 1

It’s just a slightly modified version of Ilya Grigorik’s “Desert” color theme, with some tweaks that suit my needs.

To get similar features in Leopard, we have to rely on MacFuse and sshfs. I’ll explain here how I’ve installed these components on MacOSX 10.5.

First, download the latest MacFuse dmg and install it. FYI, the version I’ve got was MacFuse 2.0.3,2.

Then, download the sshfs executable for Leopard, either the gzipped version or the binary from the SVN as explained in the MacFuse wiki.

From a terminal, rename the binary:

sudo mv ./sshfs-static-leopard ./sshfs

Then allow the binary to be executed and place it in the system:

sudo chmod +x sshfs
sudo install sshfs /usr/local/bin

From now you can test sshfs mounting with the following command:

sshfs user@myserver.net:/folder/ /Network/distant-folder -p 22

I personally had a problem here: sshfs complained about a missing library. I fixed this by downloading the required file from the MacFusion project and copying it beside the sshfs binary:

sudo wget http://www.macfusionapp.org/trac/export/86/trunk/SSHFS/sshnodelay.so
sudo mv ./sshnodelay.so /usr/local/bin/
sudo chmod +x /usr/local/bin/sshnodelay.so

If this fail you can also check:

• that the current user you’re logged with has access to the distant server with the ssh user@myserver.net command;
• or that the local mount point exists (you can create it with mkdir -p /Network/distant-folder);
• and finally, you can add the -o debug option to the sshfs command above to get additional clues.

Now we will automate the mounting of sshfs at every start.

At this point I recommend you to register the root user of your MacOSX system to the distant server:

sudo cat ~/.ssh/id_rsa.pub | sudo ssh -p 22 user@myserver.net "cat >> ~/.ssh/authorized_keys"

If doesn’t exists, we have to create the /etc/fstab to edit it:

sudo touch /etc/fstab
sudo vi /etc/fstab

And add the following directives:

dummy:user@myserver.net:/folder/ /Network/distant-folder sshfs allow_other,auto_cache,reconnect,port=22,follow_symlinks,volname="Distant folder" 0 0

As you can see I’ve added lots of options to accommodate my uses. You can get more informations about sshfs options through traditional help pages:

sshfs --help

MacOSX’s automount daemon will look for a script called mount_sshfs at start. Actually it doesn’t exists on your system, but sshfs command line is compatible with what automount expect. So creating a symbolic link will do the trick:

sudo ln -s /usr/local/bin/sshfs /sbin/mount_sshfs

Finally, we can tell automount to acknowledge all our modifications:

sudo automount -vc

Changelog:

• Check version of Python (at least v2.4 is required)
• Rename --debug option to --verbose
• Add a --dry-run option for testing
• Remove use of deprecated pexpect methods
• Add and update some error messages

The main update is about the lock file, which I implemented in the last version to keep the script to run twice (or more) in parallel. This is a nice feature to avoid overlapping processes that fight each other to use the same ressources. But in some extreme cases (reboot or power failure during backup, …), the lock file will remain and so will prevent the script to start (until you notice the problem and remove the lock file manually). This new version take care of this problem and is now able to remove the lock automatically if a timeout is reached. It also kill all remaining child processes.

Here is the detailed changelog:

• Auto-kill the script if the backup process take to much time. Timeout can be defined via a constant.
• Clean kill: track all child processes to kill them safely before removing the lock file.
• Require newer versions of python (>= v2.4), rsync (>= v2.6.7) and rdiff-backup (>= v1.1.0).
• Use --preserve-numerical-ids option when adding rdiff-backup increment.
• Keep 15 increments by default instead of 20. This value can be easily changed thanks to a defined constant.
• Remove deleted file first during mirroring and delete outdated increments before adding a new one to gain space. This strategy is safer for target disk with low remaining free space.
• Tell rsync to print human-readable values.

I run this script to backup all the local machines within my LAN. I start the backup process everyday thanks to a cron entry similar to this one:

0 20 * * * root /root/system-backup.py >> /mnt/backup-disk/backup.log

If you need more information about the rsync part the script, please have a look to my previous Remote Backup with rsync article, which detail how-to setup key authentification with ssh.

Change log:

• Each item of the user’s backup_list must specify the type property (FTP, FTPs, SSH, MySQLdump or MySQLdump+ssh).
• The property previously known as site is now host.
• File system structure changed: /ftp-mirror folders renamed to /mirror.
• Add SSH backups.
• The script is able to detect if a SSH connexion can be initiated without a password. This was designed for people who don’t like the idea of storing clear password in the script. Thanks to this feature, you can benefit public key authentication from OpenSSH.
• Use of rsync whenever it’s possible for bandwidth efficiency.
• FTP and FTPs (aka FTP over SSL) are now handled separately: this suppress the default fall-back to FTP if FTPs is not supported by the remote server. This is safer as it doesn’t let lftp make the decision for you to send your clear password on the net.
• All ports are optionnal, no need to specify it you use default ports.
• Add MySQL backups thanks to mysqldump.
• Two mode of MySQL backups: through SSH or direct connection to server.
• A particular database to backup can be specified. Else, all databases are backed up.
• Much more detailed logs that include external command’s output.
• Auto-detect the existence of required external tools and commands at launch.
• Use pexpect lib to simulate user password input.
• Run all external commands in english for consistency.
• Check that the script is running in a posix environnement.
• Fix bug related to directory creation.

If you were using a previous version of my backup script and want to use this updated version, take care of changes, especially the ones describes in the first 3 items of the change log above.

Prerequisites

• A distant server, where backup will be stored (homeserver.com in this case),
• A user account on this server (mine was kevin),
• A ssh deamon running on the server that allow the user to log in.

Setup rsync

First, install rsync on the client and on the server using:

urpmi rsync

Synchronization

Then, to synchronise from the local machine to the distant server, just do:

rsync -avz -e ssh /home/client_user/Documents kevin@homeserver.com:/mnt/raid2/

• /home/client_user/Documents is the local folder we want to save (located in the home folder of the client user client_user),
• homeserver.com is the distant server name (could be en IP address),
• kevin is the distant user,
• /mnt/raid2/ is the distant folder where we want to save the local one.

Croned synchronization

First, create a pair of cryptographic keys (public, private):

ssh-keygen -t rsa

Then, from the local machine as user client_user, register you on the distant server:

ssh-copy-id -i ~/.ssh/id_rsa.pub kevin@homeserver.com

In case your distant machine’s SSH server is running on another port than 22 (which is the default port), let’s said 222, here is the command that emulate ssh-copy-id (as the later doesn’t have a port parameter):

cat ~/.ssh/id_rsa.pub | ssh -p 222 kevin@homeserver.com "cat >> ~/.ssh/authorized_keys"

Create a script named rsync_data_backup.sh that contain the command you’ve used previously to synchronize your data:

rsync -avz -e ssh /home/client_user/Documents kevin@homeserver.com:/mnt/raid2/

To run this script with a cron entry, the (unsecure) solution found is to create a key without a passphrase. The cron entry could be something like:

15 13 * * 1-5 client_user /home/client_user/rsync_data_backup.sh > /home/client_user/rsync_data_backup.log

This crontab entry will automaticcaly synchronise our data each first-5 days of the week, at 13:15.