Difference between revisions of "Backup with rdiff-backup"

From OpenKM Documentation
Jump to: navigation, search
(→‎Remote backup with rdiff-backup (Tomcat - PostgreSQL))
(→‎Local backup to USB disk with rdiff-backup)
Line 93: Line 93:
 
# Backup and purge old backups
 
# Backup and purge old backups
 
rdiff-backup --remove-older-than 30B $BACKUP_DIR/$HOST
 
rdiff-backup --remove-older-than 30B $BACKUP_DIR/$HOST
−
rdiff-backup -v 3 --print-statistics --exclude /media --exclude /mnt \
+
rdiff-backup -v 3 --print-statistics --include $OPENKM_HOME --exclude '**' / $BACKUP_DIR/$HOST
−
  --include $OPENKM_HOME --exclude '**' / $BACKUP_DIR/$HOST
 
  
 
# Start JBoss
 
# Start JBoss

Revision as of 16:34, 13 November 2012

rdiff-backup backs up one directory to another, possibly over a network. The target directory ends up a copy of the source directory, but extra reverse diffs are stored in a special subdirectory of that target directory, so you can still recover files lost some time ago. The idea is to combine the best features of a mirror and an incremental backup. rdiff-backup also preserves subdirectories, hard links, dev files, permissions, uid/gid ownership, modification times, extended attributes, acls, and resource forks. Also, rdiff-backup can operate in a bandwidth efficient manner over a pipe, like rsync. Thus you can use rdiff-backup and ssh to securely back a hard drive up to a remote location, and only the differences will be transmitted.


Nota clasica.png rdiff-backup from Debian throws a warning due to a deprecated method [os.popen2 is deprecated], which can be hidden following the steps at How to shut up Python deprecation warnings.


Nota clasica.png If you need to backup to CIFS (SMB, Samba) or Mac's HFS, please take a look at rdiff-backup FAQ.

More info about rdiff-backup:

Remote backup with rdiff-backup

#!/bin/bash
#
## BEGIN CONFIG ##
HOST=$(uname -n)
FILES="/home/openkm"
BACKUP_DIR="/mnt/backup"
## END CONFIG ##
echo -e "### BEGIN: $(date +"%x %X") ###\n"

# Stop JBoss
/etc/init.d/jboss stop

# Backup and purge old backups
rdiff-backup --remove-older-than 30B backup@server::$BACKUP_DIR/$HOST
rdiff-backup -v 3 --print-statistics --exclude /media --exclude /mnt \
  --include $FILES --exclude '**' / backup@server::$BACKUP_DIR/$HOST

# Start JBoss
/etc/init.d/jboss start
echo -e "\n### END: $(date +"%x %X") ###"

# Status
echo "=================================";
rdiff-backup --list-increment-sizes backup@server::$BACKUP_DIR/$HOST
echo "=================================";

Local backup to USB disk with rdiff-backup

USB disk mount point can be defined in /etc/fstab as: 

 /dev/sdb1       /mnt/backup     ext4    defaults        0       0

#!/bin/bash
#
## BEGIN CONFIG ##
HOST=$(uname -n)
MYSQL_PASS=""
OPENKM_HOME="/home/openkm"
JBOSS_HOME="$OPENKM_HOME/jboss-4.2.3.GA"
DATABASE_EXP="$OPENKM_HOME/db"
BACKUP_DIR="/mnt/backup"
## END CONFIG ##

if [ $(id -u) != 0 ]; then echo "You should run this script as root"; exit; fi

echo -e "### BEGIN: $(date +"%x %X") ###\n"
rm -rf $DATABASE_EXP
mkdir -p $DATABASE_EXP

# Mount disk
if mount | grep "$BACKUP_DIR type" > /dev/null; then
  echo "$BACKUP_DIR already mounted";
else
  mount "$BACKUP_DIR";
fi

# Stop JBoss
/etc/init.d/jboss stop

# Clean logs
echo "Clean JBoss temporal files."
# rm -rf $JBOSS_HOME/server/default/log
rm -rf $JBOSS_HOME/server/default/tmp
rm -rf $JBOSS_HOME/server/default/work
rm -rf $JBOSS_HOME/server/default/data/wsdl

# Backup de MySQL
if [ -n "$MYSQL_PASS" ]; then
  MYSQL_DBS=$(mysqlshow -h localhost -u root -p$MYSQL_PASS | awk '(NR > 2) && (/[a-zA-Z0-9]+[ ]+[|]/) && ($2 != "mysql") && ($2 != "test") && ($2 != "information_schema") { print $2 }');
  
  for DB in $MYSQL_DBS ; do
    echo "* Backuping MySQL data from $DB...";
    mysqldump -h localhost -u root -p$MYSQL_PASS $DB > $DATABASE_EXP/mysql_$DB.sql
  done
  echo "-------------------------------------";
fi

# Backup and purge old backups
rdiff-backup --remove-older-than 30B $BACKUP_DIR/$HOST
rdiff-backup -v 3 --print-statistics --include $OPENKM_HOME --exclude '**' / $BACKUP_DIR/$HOST

# Start JBoss
/etc/init.d/jboss start
echo -e "\n### END: $(date +"%x %X") ###"

# Status
echo "=================================";
rdiff-backup --list-increment-sizes $BACKUP_DIR/$HOST
echo "*********************************";
df -h | grep "$BACKUP_DIR"
echo "=================================";

# Umount disk
sync
umount "$BACKUP_DIR"

Be sure to have the last JBoss startup script, otherwise use this piece of code to ensure that JBoss is stopped before backuping the files: 

while [ "$(ps -ef | grep java | grep jboss | wc -l)" -gt "0" ]; do
  sleep 5; echo ".";
done

Remote backup with rdiff-backup (JBoss)

#!/bin/bash
#
# @weekly /root/backup.sh
#
## BEGIN CONFIG ##
HOST=$(uname -n)
MYSQL_PASS="****"
OPENKM_HOME="/home/openkm"
JBOSS_HOME="$OPENKM_HOME/jboss-4.2.3.GA"
DATABASE_EXP="$OPENKM_HOME/db"
BACKUP_DST="user@server::/path/to/backup"
## END CONFIG ##

echo -e "### BEGIN: $(date +"%x %X") ###\n"
rm -rf $DATABASE_EXP
mkdir -p $DATABASE_EXP
 
# Stop JBoss
/etc/init.d/jboss stop

# Clean logs
echo "Clean JBoss temporal files."
# rm -rf $JBOSS_HOME/server/default/log
rm -rf $JBOSS_HOME/server/default/tmp
rm -rf $JBOSS_HOME/server/default/work
rm -rf $JBOSS_HOME/server/default/data/wsdl

# Backup de MySQL
MYSQL_DBS=$(mysqlshow -h localhost -u root -p$MYSQL_PASS | awk '(NR > 2) && (/[a-zA-Z0-9]+[ ]+[|]/) && ($2 != "mysql") && ($2 != "test") && ($2 != "information_schema") { print $2 }');

for DB in $MYSQL_DBS ; do
  echo "* Backuping MySQL data from $DB..."
  mysqldump -h localhost -u root -p$MYSQL_PASS $DB > $DATABASE_EXP/mysql_$DB.sql
done

echo "-------------------------------------"
 
# Backup and purge old backups
rdiff-backup --remove-older-than 30B $BACKUP_DST
rdiff-backup -v 3 --print-statistics --include $OPENKM_HOME --exclude '**' / $BACKUP_DST
 
# Start JBoss
/etc/init.d/jboss start
echo -e "\n### END: $(date +"%x %X") ###"
 
# Status
echo "=================================";
rdiff-backup --list-increment-sizes $BACKUP_DST
echo "=================================";

Remote backup with rdiff-backup (Tomcat - MySQL)

#!/bin/bash
#
# @weekly /root/backup.sh
#
## BEGIN CONFIG ##
HOST=$(uname -n)
MYSQL_PASS="****"
OPENKM_HOME="/home/openkm"
TOMCAT_HOME="$OPENKM_HOME/tomcat-7.0.27"
DATABASE_EXP="$OPENKM_HOME/db"
BACKUP_DST="user@server::/path/to/backup"
## END CONFIG ##

echo -e "### BEGIN: $(date +"%x %X") ###\n"
rm -rf $DATABASE_EXP
mkdir -p $DATABASE_EXP
 
# Stop Tomcat
/etc/init.d/tomcat stop

# Clean logs
#echo "Clean Tomcat temporal files."
#rm -rf $TOMCAT_HOME/logs/*
#rm -rf $TOMCAT_HOME/temp/*
#rm -rf $TOMCAT_HOME/work/Catalina/localhost

# Backup de MySQL
MYSQL_DBS=$(mysqlshow -h localhost -u root -p$MYSQL_PASS | awk '(NR > 2) && (/[a-zA-Z0-9]+[ ]+[|]/) && ($2 != "mysql") && ($2 != "test") && ($2 != "information_schema") { print $2 }');

for DB in $MYSQL_DBS ; do
  echo "* Backuping MySQL data from $DB..."
  mysqldump -h localhost -u root -p$MYSQL_PASS $DB > $DATABASE_EXP/mysql_$DB.sql
done

echo "-------------------------------------"
 
# Backup and purge old backups
rdiff-backup --remove-older-than 30B $BACKUP_DST
rdiff-backup -v 3 --print-statistics --include $OPENKM_HOME --exclude '**' / $BACKUP_DST
 
# Start Tomcat
/etc/init.d/tomcat start
echo -e "\n### END: $(date +"%x %X") ###"
 
# Status
echo "=================================";
rdiff-backup --list-increment-sizes $BACKUP_DST
echo "=================================";

Remote backup with rdiff-backup (Tomcat - PostgreSQL)

#!/bin/bash
#
# @weekly /root/backup.sh
#
## BEGIN CONFIG ##
HOST=$(uname -n)
OPENKM_HOME="/home/openkm"
TOMCAT_HOME="$OPENKM_HOME/tomcat-7.0.27"
DATABASE_EXP="$OPENKM_HOME/db"
BACKUP_DST="user@server::/path/to/backup"
## END CONFIG ##

echo -e "### BEGIN: $(date +"%x %X") ###\n"
rm -rf $DATABASE_EXP
mkdir -p $DATABASE_EXP
 
# Stop Tomcat
/etc/init.d/tomcat stop

# Clean logs
#echo "Clean Tomcat temporal files."
#rm -rf $TOMCAT_HOME/logs/*
#rm -rf $TOMCAT_HOME/temp/*
#rm -rf $TOMCAT_HOME/work/Catalina/localhost

# Backup de PostgreSQL
POSTGRESQL_DBS=$(su postgres -c "psql -l" | awk '(NR > 2) && (/[a-zA-Z0-9]+[ ]+[|]/) && ($1 !~ /template[0-1]/) && ($1 != "postgres") { print $1 }');

for DB in $POSTGRESQL_DBS ; do
  echo "* Backuping PostgreSQL data from $DB..."
  su postgres -c "pg_dump $DB" > $DATABASE_EXP/pg_$DB.sql
done

# Databases optimizations
su postgres -c "vacuumdb -a -z" > /dev/null
su postgres -c "reindexdb -a -q" 2> /dev/null

echo "-------------------------------------"
 
# Backup and purge old backups
rdiff-backup --remove-older-than 30B $BACKUP_DST
rdiff-backup -v 3 --print-statistics --include $OPENKM_HOME --exclude '**' / $BACKUP_DST
 
# Start Tomcat
/etc/init.d/tomcat start
echo -e "\n### END: $(date +"%x %X") ###"
 
# Status
echo "=================================";
rdiff-backup --list-increment-sizes $BACKUP_DST
echo "=================================";
Retrieved from "https://www.openkm.com/wiki/index.php?title=Backup_with_rdiff-backup&oldid=7769"