Tag Archive for 'ssh'

Fuse and sshfs on MacOSX Leopard

I’m used to access distant machine’s file systems via ssh. My favorite environment, KDE, makes things easy thanks to the support of sftp:// URLs via a kio_slave. MacOSX is not as friendly and don’t have any built-in mechanism of that kind.

To get similar features in Leopard, we have to rely on MacFuse and sshfs. I’ll explain here how I’ve installed these components on MacOSX 10.5.

MacFUSE_Banner

First, download the latest MacFuse dmg and install it. FYI, the version I’ve got was MacFuse 2.0.3,2.

Then, download the sshfs executable for Leopard, either the gzipped version or the binary from the SVN as explained in the MacFuse wiki.

From a terminal, rename the binary:

sudo mv ./sshfs-static-leopard ./sshfs

Then allow the binary to be executed and place it in the system:

sudo chmod +x sshfs
sudo install sshfs /usr/local/bin

From now you can test sshfs mounting with the following command:

sshfs user@myserver.net:/folder/ /Network/distant-folder -p 22

I personally had a problem here: sshfs complained about a missing library. I fixed this by downloading the required file from the MacFusion project and copying it beside the sshfs binary:

sudo wget http://www.macfusionapp.org/trac/export/86/trunk/SSHFS/sshnodelay.so
sudo mv ./sshnodelay.so /usr/local/bin/
sudo chmod +x /usr/local/bin/sshnodelay.so

If this fail you can also check:

  • that the current user you’re logged with has access to the distant server with the ssh user@myserver.net command;
  • or that the local mount point exists (you can create it with mkdir -p /Network/distant-folder);
  • and finally, you can add the -o debug option to the sshfs command above to get additional clues.

Now we will automate the mounting of sshfs at every start.

At this point I recommend you to register the root user of your MacOSX system to the distant server:

sudo cat ~/.ssh/id_rsa.pub | sudo ssh -p 22 user@myserver.net "cat >> ~/.ssh/authorized_keys"

If doesn’t exists, we have to create the /etc/fstab to edit it:

sudo touch /etc/fstab
sudo vi /etc/fstab

And add the following directives:

dummy:user@myserver.net:/folder/ /Network/distant-folder sshfs allow_other,auto_cache,reconnect,port=22,follow_symlinks,volname="Distant folder" 0 0

As you can see I’ve added lots of options to accommodate my uses. You can get more informations about sshfs options through traditional help pages:

sshfs --help

MacOSX’s automount daemon will look for a script called mount_sshfs at start. Actually it doesn’t exists on your system, but sshfs command line is compatible with what automount expect. So creating a symbolic link will do the trick:

sudo ln -s /usr/local/bin/sshfs /sbin/mount_sshfs

Finally, we can tell automount to acknowledge all our modifications:

sudo automount -vc

Website Backup Script: bug fix release

14 months after the last release, here is a new version of my website backup script. As you can see in the changelog, this version is essentially released to fix some bugs.

Changelog:

  • Check version of Python (at least v2.4 is required)
  • Rename --debug option to --verbose
  • Add a --dry-run option for testing
  • Remove use of deprecated pexpect methods
  • Add and update some error messages

System backup script: no more endless lock

I’ve just released a new version of my system-backup.py script.

The main update is about the lock file, which I implemented in the last version to keep the script to run twice (or more) in parallel. This is a nice feature to avoid overlapping processes that fight each other to use the same ressources. But in some extreme cases (reboot or power failure during backup, …), the lock file will remain and so will prevent the script to start (until you notice the problem and remove the lock file manually). This new version take care of this problem and is now able to remove the lock automatically if a timeout is reached. It also kill all remaining child processes.

Here is the detailed changelog:

  • Auto-kill the script if the backup process take to much time. Timeout can be defined via a constant.
  • Clean kill: track all child processes to kill them safely before removing the lock file.
  • Require newer versions of python (>= v2.4), rsync (>= v2.6.7) and rdiff-backup (>= v1.1.0).
  • Use --preserve-numerical-ids option when adding rdiff-backup increment.
  • Keep 15 increments by default instead of 20. This value can be easily changed thanks to a defined constant.
  • Remove deleted file first during mirroring and delete outdated increments before adding a new one to gain space. This strategy is safer for target disk with low remaining free space.
  • Tell rsync to print human-readable values.

System Backup on Unreliable Link thanks to rdiff-backup and rsync

I’ve just write a brand new script called system-backup.py. It’s similar to my website-backup.py script but instead of website and MySQL databases, it is designed to backup systems of several machines. This script is based on an idea from the “Backup up on unreliable link” article from the official rdiff-backup wiki. It use rdiff-backup to keep the last 20 backups and rsync to speed-up the backup process.

I run this script to backup all the local machines within my LAN. I start the backup process everyday thanks to a cron entry similar to this one:

0 20 * * * root /root/system-backup.py >> /mnt/backup-disk/backup.log

If you need more information about the rsync part the script, please have a look to my previous Remote Backup with rsync article, which detail how-to setup key authentification with ssh.

Website Backup Script: MySQL dumps and SSH supported.

Three months after the last version, here is a big update of my backup scripts for websites. The script was greatly improved and among new features, the most important is the support of backups over SSH and backups of MySQL databases.

Change log:

  • Each item of the user’s backup_list must specify the type property (FTP, FTPs, SSH, MySQLdump or MySQLdump+ssh).
  • The property previously known as site is now host.
  • File system structure changed: /ftp-mirror folders renamed to /mirror.
  • Add SSH backups.
  • The script is able to detect if a SSH connexion can be initiated without a password. This was designed for people who don’t like the idea of storing clear password in the script. Thanks to this feature, you can benefit public key authentication from OpenSSH.
  • Use of rsync whenever it’s possible for bandwidth efficiency.
  • FTP and FTPs (aka FTP over SSL) are now handled separately: this suppress the default fall-back to FTP if FTPs is not supported by the remote server. This is safer as it doesn’t let lftp make the decision for you to send your clear password on the net.
  • All ports are optionnal, no need to specify it you use default ports.
  • Add MySQL backups thanks to mysqldump.
  • Two mode of MySQL backups: through SSH or direct connection to server.
  • A particular database to backup can be specified. Else, all databases are backed up.
  • Much more detailed logs that include external command’s output.
  • Auto-detect the existence of required external tools and commands at launch.
  • Use pexpect lib to simulate user password input.
  • Run all external commands in english for consistency.
  • Check that the script is running in a posix environnement.
  • Fix bug related to directory creation.

If you were using a previous version of my backup script and want to use this updated version, take care of changes, especially the ones describes in the first 3 items of the change log above.