14 months after the last release, here is a new version of my website backup script. As you can see in the changelog, this version is essentially released to fix some bugs.
Changelog:
--debug option to --verbose--dry-run option for testingpexpect methodsThree months after the last version, here is a big update of my backup scripts for websites. The script was greatly improved and among new features, the most important is the support of backups over SSH and backups of MySQL databases.
Change log:
backup_list must specify the type property (FTP, FTPs, SSH, MySQLdump or MySQLdump+ssh).site is now host./ftp-mirror folders renamed to /mirror.rsync whenever it’s possible for bandwidth efficiency.lftp make the decision for you to send your clear password on the net.mysqldump.pexpect lib to simulate user password input.If you were using a previous version of my backup script and want to use this updated version, take care of changes, especially the ones describes in the first 3 items of the change log above.
I’ve updated my website-backup python script.
Change log:
set ftp:list-options -a command to force lftp to download hidden files (like .htaccess and so on).--force parameter to allow auto-deletion of multiple outdated rdiff-backup increments.I also added a debug mode as suggested by Sacha.
I’ve changed my backup strategy today, so I updated my website-backup.py script. You can find the latest version of the script on my script page.
I now use rdiff-backup in this script to keep 32 days of incremental backups. Beside this the script do a monthly full archive of the website in bzip2 format. This new strategy has reduced the total size of my backups from 64 GB to 6.7 GB. Roughly 90% of free space gain thanks to rdiff-backup ! If rdiff-backup is so efficient in my case, this is due to the existence on my websites of large files that are rarely modified (Mp3s, Flacs, RPMs, images, etc…).
I’ve updated my website-backup.py script. I added a little optimization to delete the yesterday’s backup if nothing was changed on the remote website. This let me save some megabytes on the hard drive for everyday backups of near-static websites. The optimization I added is simply based on checksum comparison. This is the context to the previous script I wrote today: it was a tool to help me debug and experiment this new feature.
You can find the latest version of the website-backup.py script in my Linux script page. Here is the direct link to today’s version.
Here is a little log of a micro lftp session. I post it just to remind me some basic commands…
Connect to an ftp server and login as myuser:
[kevin@localhost ~]$ lftp ftp://ftp.my-domain.com lftp ftp.my-domain.com:~> user myuser Password:
Get a list of basic commands:
lftp myuser@ftp.my-domain.com:/www> help
!<shell -command> (commands)
alias [<name> [<value>]] anon
bookmark [SUBCMD] cache [SUBCMD]
cat [-b] <files> cd <rdir>
chmod [OPTS] mode file... close [-a]
[re]cls [opts] [path/][pattern] debug [<level>|off] [-o <file>]
du [options] <dirs> exit [<code>|bg]
get [OPTS] <rfile> [-o <lfile>] glob [OPTS] <cmd> <args>
help [<cmd>] history -w file|-r file|-c|-l [cnt]
jobs [-v] kill all|<job_no>
lcd <ldir> lftp [OPTS] <site>
ls [<args>] mget [OPTS] <files>
mirror [OPTS] [remote [local]] mkdir [-p] <dirs>
module name [args] more <files>
mput [OPTS] </files><files> mrm </files><files>
mv <file1> <file2> [re]nlist [<args>]
open [OPTS] <site> pget [OPTS] <rfile> [-o <lfile>]
put [OPTS] </lfile><lfile> [-o <rfile>] pwd [-p]
queue [OPTS] [<cmd>] quote </cmd><cmd>
repeat [delay] [command] rm [-r] [-f] <files>
rmdir [-f] <dirs> scache [<session_no>]
set [OPT] [<var> [<val>]] site <site_cmd>
source <file> user <user |URL> [<pass>]
version wait [<jobno>]
zcat <files> zmore
Navigate in the file structure:
lftp myuser@ftp.my-domain.com:~> ls drwx---r-x 2 myuser users 4096 Jan 9 2005 cgi-bin drwx---r-x 2 myuser users 4096 Jan 9 2005 sessions drwx---r-x 12 myuser users 4096 Jun 1 01:44 www lftp myuser@ftp.my-domain.com:/> cd www lftp myuser@ftp.my-domain.com:/www> ls -rwx---r-x 1 myuser users 32724 Aug 17 00:43 class.php -rwx---r-x 1 myuser users 17896 Jan 10 2005 download.php -rwx---r-x 1 myuser users 168 Jan 10 2005 .htaccess drwxr-xr-x 3 myuser users 4096 Jan 10 2005 main_admin drwxr-xr-x 39 myuser users 4096 Aug 6 01:02 main_plugins
Upload file from local machine to ftp server:
lftp myuser@ftp.my-domain.com:/www> put /home/kevin/class.php 64714 bytes transferred in 8 seconds (7.9K/s)
Navigate, view, upload and exit:
lftp myuser@ftp.my-domain.com:/www> ls -rwx---r-x 1 myuser users 64714 Aug 17 00:56 class.php -rwx---r-x 1 myuser users 17896 Jan 10 2005 download.php -rwx---r-x 1 myuser users 168 Jan 10 2005 .htaccess drwxr-xr-x 3 myuser users 4096 Jan 10 2005 main_admin drwxr-xr-x 39 myuser users 4096 Aug 6 01:02 main_plugins lftp myuser@ftp.my-domain.com:/www> cd mai main_admin/ main_plugins/ lftp myuser@ftp.my-domain.com:/www> cd main_admin/ cd ok, cwd=/www/main_admin lftp myuser@ftp.my-domain.com:/www/main_admin> ls -rwx---r-x 1 myuser users 13449 Jan 10 2005 upload.php -rwx---r-x 1 myuser users 100 Aug 17 00:36 version.php -rwx---r-x 1 myuser users 4494 Jan 10 2005 message.php lftp myuser@ftp.my-domain.com:/www/main_admin> cat ./version.php < ?php $globinfo['main_version'] = "v 0.0.1 (alpha)"; $globinfo['main_build'] = "20050815"; 100 bytes transferred lftp myuser@ftp.my-domain.com:/www/main_admin> put /home/kevin/ .bash_history .bash_logout .bash_profile .bashrc .lftp/ .ssh/ .viminfo class.php main_admin/ tmp/ lftp myuser@ftp.my-domain.com:/www/main_admin> put /home/kevin/main_admin/version.php 100 bytes transferred lftp myuser@ftp.my-domain.com:/www/main_admin> ls -rwx---r-x 1 myuser users 13449 Jan 10 2005 upload.php -rwx---r-x 1 myuser users 100 Aug 17 00:58 version.php -rwx---r-x 1 myuser users 4494 Jan 10 2005 message.php lftp myuser@ftp.my-domain.com:/www/main_admin> exit [kevin@localhost ~]$
