Tag Archives: Backup

System Backup on Unreliable Link thanks to rdiff-backup and rsync

Posted on by
3

I’ve just write a brand new script called system-backup.py. It’s similar to my website-backup.py script but instead of website and MySQL databases, it is designed to backup systems of several machines. This script is based on an idea from the “Backup up on unreliable link” article from the official rdiff-backup wiki. It use rdiff-backup to keep the last 20 backups and rsync to speed-up the backup process.

I run this script to backup all the local machines within my LAN. I start the backup process everyday thanks to a cron entry similar to this one:

0 20 * * * root /root/system-backup.py >> /mnt/backup-disk/backup.log

If you need more information about the rsync part the script, please have a look to my previous Remote Backup with rsync article, which detail how-to setup key authentification with ssh.

Website Backup Script: MySQL dumps and SSH supported.

Posted on by
9

Three months after the last version, here is a big update of my backup scripts for websites. The script was greatly improved and among new features, the most important is the support of backups over SSH and backups of MySQL databases.

Change log:

  • Each item of the user’s backup_list must specify the type property (FTP, FTPs, SSH, MySQLdump or MySQLdump+ssh).
  • The property previously known as site is now host.
  • File system structure changed: /ftp-mirror folders renamed to /mirror.
  • Add SSH backups.
  • The script is able to detect if a SSH connexion can be initiated without a password. This was designed for people who don’t like the idea of storing clear password in the script. Thanks to this feature, you can benefit public key authentication from OpenSSH.
  • Use of rsync whenever it’s possible for bandwidth efficiency.
  • FTP and FTPs (aka FTP over SSL) are now handled separately: this suppress the default fall-back to FTP if FTPs is not supported by the remote server. This is safer as it doesn’t let lftp make the decision for you to send your clear password on the net.
  • All ports are optionnal, no need to specify it you use default ports.
  • Add MySQL backups thanks to mysqldump.
  • Two mode of MySQL backups: through SSH or direct connection to server.
  • A particular database to backup can be specified. Else, all databases are backed up.
  • Much more detailed logs that include external command’s output.
  • Auto-detect the existence of required external tools and commands at launch.
  • Use pexpect lib to simulate user password input.
  • Run all external commands in english for consistency.
  • Check that the script is running in a posix environnement.
  • Fix bug related to directory creation.

If you were using a previous version of my backup script and want to use this updated version, take care of changes, especially the ones describes in the first 3 items of the change log above.

How-to Recover a RAID array after having Zero-ized Superblocks

Posted on by
34

Today mdadm send me a mail to warn that one of my hard drive (/dev/hdd1) was ejected from my RAID-5 array. After some manipulations (no writes, just reads on the file system to get informations) and reboots, I ended up with a file system in a strange state: the folder structure was totally messed up and lots of files disappeared.

Assuming that this situation was about an inconsistent file index, I decided to reset the superblocks of the remaining physical disks:

mdadm --zero-superblock /dev/hdc1
mdadm --zero-superblock /dev/hdb1

I don’t know why I decided to do so, but it was the stupidest idea of the week. After such a violent treatment, my array refused to start:

[root@localhost ~]$ mdadm --assemble /dev/md0 --auto --scan --update=summaries --verbose
mdadm: looking for devices for /dev/md0
mdadm: no RAID superblock on /dev/hdc1
mdadm: /dev/hdc1 has wrong raid level.
mdadm: no RAID superblock on /dev/hdb1
mdadm: /dev/hdb1 has wrong raid level.
mdadm: no devices found for /dev/md0

At this moment I was sure that all my data assets were lost. I was desperate. My only alternative was to ask Google. So I did.

I spend several minutes browsing the web without hope. I finally found someone in the same situation as mine (sorry, in french) on debian-user-french mailing list.

The solution was to recreate the RAID array. This sound counter-intuitive: if we recreate a raid array over an existing one, it will be erased ! Right ? Wrong ! As it is said on debian-user-french, mdadm is smart enough to “see” that HDD of the new array were elements of a previous one. Knowing that, mdadm will try to do its best (i.e. if parameters match the previous array configuration) and rebuild the new array upon the previous one in a non-destructive way, by keeping HDD content.

So, here is how I finally recovered my RAID array:

[root@localhost ~]$ mdadm --create /dev/md0 --verbose --level=5 --raid-devices=3 /dev/hdc1 missing /dev/hdb1
mdadm: layout defaults to left-symmetric
mdadm: chunk size defaults to 64K
mdadm: size set to 312568576K
mdadm: array /dev/md0 started.

Of course this doesn’t solve my initial problem about the /dev/md0 file system: it is still in an altered state. Maybe it’s too late to recover data. But at least I reverted all my today’s mistakes, and the situation will not deteriorate until I power up my RAID ! :)

Website Backup Script Updated: Take Care of Hidden Files now.

Posted on by
1

I’ve updated my website-backup python script.

Change log:

  • Use set ftp:list-options -a command to force lftp to download hidden files (like .htaccess and so on).
  • Use --force parameter to allow auto-deletion of multiple outdated rdiff-backup increments.
  • Defensive incremental backup policy: keep 32 last backups instead of 32 days of backup.

I also added a debug mode as suggested by Sacha.

Website Backup script: Incremental Backup feature added.

Posted on by
6

I’ve changed my backup strategy today, so I updated my website-backup.py script. You can find the latest version of the script on my script page.

I now use rdiff-backup in this script to keep 32 days of incremental backups. Beside this the script do a monthly full archive of the website in bzip2 format. This new strategy has reduced the total size of my backups from 64 GB to 6.7 GB. Roughly 90% of free space gain thanks to rdiff-backup ! If rdiff-backup is so efficient in my case, this is due to the existence on my websites of large files that are rarely modified (Mp3s, Flacs, RPMs, images, etc…).

Website Backup script: New Version Save you Disk Space.

Posted on by
2

I’ve updated my website-backup.py script. I added a little optimization to delete the yesterday’s backup if nothing was changed on the remote website. This let me save some megabytes on the hard drive for everyday backups of near-static websites. The optimization I added is simply based on checksum comparison. This is the context to the previous script I wrote today: it was a tool to help me debug and experiment this new feature.

You can find the latest version of the website-backup.py script in my Linux script page. Here is the direct link to today’s version.