Difference between revisions of "Backup scripts"

From OpenKM Documentation
Jump to: navigation, search
(Splitted in several articles)
Line 25: Line 25:
 
For more information take a look at [http://adminschoice.com/crontab-quick-reference Crontab quick reference]
 
For more information take a look at [http://adminschoice.com/crontab-quick-reference Crontab quick reference]
  
−
== Remote backup with rsync ==
+
* [[Backup with rsync]] [[File:Padlock.gif]]
−
<source lang="bash">
+
* [[Backup with rdiff-backup]] [[File:Padlock.gif]]
−
#!/bin/bash
+
* [[Backup with duplicity]] [[File:Padlock.gif]]
−
#
 
−
## BEGIN CONFIG ##
 
−
HOST=$(uname -n)
 
−
FILES="/home/openkm"
 
−
BACKUP_DIR="/mnt/backup"
 
−
## END CONFIG ##
 
−
echo -e "### BEGIN: $(date +"%x %X") ###\n"
 
−
 
 
−
# Stop JBoss
 
−
/etc/init.d/jboss stop
 
−
while [ "$(ps -ef | grep java | grep jboss | wc -l)" -gt "0" ]; do
 
−
  sleep 5; echo ".";
 
−
done
 
−
 
 
−
# Copy to backup server
 
−
rsync -apzhR --stats --delete --exclude=*~ --delete-excluded $FILES backup@server:$BACKUP_DIR/$HOST
 
−
 
 
−
# Start JBoss
 
−
/etc/init.d/jboss start
 
−
echo -e "\n### END: $(date +"%x %X") ###"
 
−
</source>
 
−
 
 
−
== Remote backup with rsync and rotation ==
 
−
<source lang="bash">
 
−
#!/bin/bash
 
−
#
 
−
## BEGIN CONFIG ##
 
−
HOST=$(uname -n)
 
−
FILES="/home/openkm"
 
−
BACKUP_DIR="/mnt/backup"
 
−
## END CONFIG ##
 
−
echo -e "### BEGIN: $(date +"%x %X") ###\n"
 
−
 
 
−
# Stop JBoss
 
−
/etc/init.d/jboss stop
 
−
 
 
−
# Copy to backup server
 
−
ssh backup@server "cd $HOST; rm -rf backup.3; mv backup.2 backup.3; mv backup.1 backup.2; mv backup.0 backup.1"
 
−
rsync -apzhR --stats --delete --exclude=*~ --delete-excluded --link-dest="$BACKUP_DIR/$HOST/backup.1" \
 
−
$FILES backup@server:$BACKUP_DIR/$HOST/backup.0
 
−
 
 
−
# Start JBoss
 
−
/etc/init.d/jboss start
 
−
echo -e "\n### END: $(date +"%x %X") ###"
 
−
</source>
 
−
 
 
−
== Local backup with rsync and rotation ==
 
−
<source lang="bash">
 
−
#!/bin/bash
 
−
#
 
−
## BEGIN CONFIG ##
 
−
HOST=$(uname -n)
 
−
FILES="/home/openkm"
 
−
BACKUP_DIR="/mnt/backup"
 
−
## END CONFIG ##
 
−
echo -e "### BEGIN: $(date +"%x %X") ###\n"
 
−
 
 
−
# Stop JBoss
 
−
/etc/init.d/jboss stop
 
−
 
 
−
# Calculate snapshot
 
−
LAST_SNAPSHOT=`ls -ltr $BACKUP_DIR | tail -1 | awk {'print $8'} | cut -d . -f 2`
 
−
NEW_SNAPSHOT=$((LAST_SNAPSHOT+1))
 
−
 
 
−
# Copy to backup server
 
−
rsync -apzhR --stats --delete --exclude=*~ --exclude="$JBOSS_HOME/cache" --delete-excluded \
 
−
--link-dest="$BACKUP_DIR/$HOST/backup.$LAST_SNAPSHOT" $FILES "$BACKUP_DIR/$HOST/backup.$NEW_SNAPSHOT"
 
−
 
 
−
# Start JBoss
 
−
/etc/init.d/jboss start
 
−
echo -e "\n### END: $(date +"%x %X") ###"
 
−
 
 
−
# Status
 
−
echo "=================================";
 
−
du -hs $BACKUP_DIR
 
−
echo "*********************************";
 
−
du -hs --time $BACKUP_DIR/*
 
−
echo "=================================";
 
−
df -h | grep "$BACKUP_DIR"
 
−
echo "=================================";
 
−
</source>
 
−
 
 
−
== Remote backup with rdiff-backup ==
 
−
{{Note|rdiff-backup from Debian throws a warning due to a deprecated method [[http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=590107 os.popen2 is deprecated]], which can be hidden following the steps at [http://www.linuxinsight.com/how-to-shut-up-python-deprecation-warnings.html How to shut up Python deprecation warnings].}}
 
−
 
 
−
<source lang="bash">
 
−
#!/bin/bash
 
−
#
 
−
## BEGIN CONFIG ##
 
−
HOST=$(uname -n)
 
−
FILES="/home/openkm"
 
−
BACKUP_DIR="/mnt/backup"
 
−
## END CONFIG ##
 
−
echo -e "### BEGIN: $(date +"%x %X") ###\n"
 
−
 
 
−
# Stop JBoss
 
−
/etc/init.d/jboss stop
 
−
 
 
−
# Backup and purge old backups
 
−
rdiff-backup --remove-older-than 30B backup@server::$BACKUP_DIR/$HOST
 
−
rdiff-backup -v 3 --print-statistics --exclude /media --exclude /mnt \
 
−
  --include $FILES --exclude '**' / backup@server::$BACKUP_DIR/$HOST
 
−
 
 
−
# Start JBoss
 
−
/etc/init.d/jboss start
 
−
echo -e "\n### END: $(date +"%x %X") ###"
 
−
 
 
−
# Status
 
−
echo "=================================";
 
−
rdiff-backup --list-increment-sizes backup@server::$BACKUP_DIR/$HOST
 
−
echo "=================================";
 
−
</source>
 
−
 
 
−
More info about rdiff-backup:
 
−
* [http://rdiff-backup.nongnu.org/ rdiff-backup home page]
 
−
* [http://wiki.rdiff-backup.org/wiki/index.php/Main_Page rdiff-backup wiki]
 
−
 
 
−
== Local backup to USB disk with rdiff-backup ==
 
−
USB disk mount point can be defined in ''/etc/fstab'' as:
 
−
 
 
−
  /dev/sdb1      /mnt/backup    ext4    defaults        0      0
 
−
 
 
−
<source lang="bash">
 
−
#!/bin/bash
 
−
#
 
−
## BEGIN CONFIG ##
 
−
HOST=$(uname -n)
 
−
MYSQL_PASS=""
 
−
OPENKM_HOME="/home/openkm"
 
−
JBOSS_HOME="$OPENKM_HOME/jboss-4.2.3.GA"
 
−
DATABASE_EXP="$OPENKM_HOME/db"
 
−
BACKUP_DIR="/mnt/backup"
 
−
## END CONFIG ##
 
−
 
 
−
if [ $(id -u) != 0 ]; then echo "You should run this script as root"; exit; fi
 
−
 
 
−
echo -e "### BEGIN: $(date +"%x %X") ###\n"
 
−
rm -rf $DATABASE_EXP
 
−
mkdir -p $DATABASE_EXP
 
−
 
 
−
# Mount disk
 
−
if mount | grep "$BACKUP_DIR type" > /dev/null; then
 
−
  echo "$BACKUP_DIR already mounted";
 
−
else
 
−
  mount $BACKUP_DIR;
 
−
fi
 
−
 
 
−
# Stop JBoss
 
−
/etc/init.d/jboss stop
 
−
 
 
−
# Clean logs
 
−
echo "Clean JBoss temporal files."
 
−
rm -rf $JBOSS_HOME/server/default/log
 
−
rm -rf $JBOSS_HOME/server/default/tmp
 
−
rm -rf $JBOSS_HOME/server/default/work
 
−
 
 
−
# Backup de MySQL
 
−
if [ -n "$MYSQL_PASS" ]; then
 
−
  MYSQL_DBS=$(mysqlshow -h localhost -u root -p$MYSQL_PASS | awk '(NR > 2) && (/[a-zA-Z0-9]+[ ]+[|]/) && ($2 != "mysql") && ($2 != "test") && ($2 != "information_schema") { print $2 }');
 
−
 
 
−
  for DB in $MYSQL_DBS ; do
 
−
    echo "* Backuping MySQL data from $DB...";
 
−
    mysqldump -h localhost -u root -p$MYSQL_PASS $DB > $DATABASE_EXP/mysql_$DB.sql
 
−
  done
 
−
  echo "-------------------------------------";
 
−
fi
 
−
 
 
−
# Backup and purge old backups
 
−
rdiff-backup --remove-older-than 30B $BACKUP_DIR/$HOST
 
−
rdiff-backup -v 3 --print-statistics --exclude /media --exclude /mnt \
 
−
  --include $OPENKM_HOME --exclude '**' / $BACKUP_DIR/$HOST
 
−
 
 
−
# Start JBoss
 
−
/etc/init.d/jboss start
 
−
echo -e "\n### END: $(date +"%x %X") ###"
 
−
 
 
−
# Status
 
−
echo "=================================";
 
−
rdiff-backup --list-increment-sizes $BACKUP_DIR/$HOST
 
−
echo "*********************************";
 
−
df -h | grep "$BACKUP_DIR"
 
−
echo "=================================";
 
−
 
 
−
# Umount disk
 
−
sync
 
−
umount $BACKUP_DIR
 
−
</source>
 
−
 
 
−
Be sure to have the last JBoss startup script, otherwise use this piece of code to ensure that JBoss is stopped before backuping the files:
 
−
 
 
−
<source lang="bash">
 
−
while [ "$(ps -ef | grep java | grep jboss | wc -l)" -gt "0" ]; do
 
−
  sleep 5; echo ".";
 
−
done
 
−
</source>
 
−
 
 
−
== Backup with duplicity ==
 
−
Duplicity backs directories by producing encrypted tar-format volumes and uploading them to a remote or local file server. Because duplicity uses librsync, the incremental archives are space efficient and only record the parts of files that have changed since the last backup. Because duplicity uses GnuPG to encrypt and/or sign these archives, they will be safe from spying and/or modification by the server.
 
−
 
 
−
Duplicity can be installed in Debian / Ubuntu as simple as:
 
−
 
 
−
  $ sudo aptitude install duplicity
 
−
 
 
−
But it not in the CentOS / RedHat default repositories, so you need to install from another source. This script will help in this installation process:
 
−
 
 
−
<source lang="bash">
 
−
#!/bin/bash
 
−
#ARCH=i386
 
−
ARCH=x86_64
 
−
 
 
−
wget http://download.fedora.redhat.com/pub/epel/5/$ARCH/duplicity-0.6.14-1.el5.$ARCH.rpm
 
−
wget http://download.fedora.redhat.com/pub/epel/5/$ARCH/ncftp-3.2.2-1.el5.$ARCH.rpm
 
−
wget http://download.fedora.redhat.com/pub/epel/5/$ARCH/librsync-0.9.7-13.el5.$ARCH.rpm
 
−
wget http://download.fedora.redhat.com/pub/epel/5/$ARCH/python-GnuPGInterface-0.3.2-2.el5.noarch.rpm
 
−
wget http://download.fedora.redhat.com/pub/epel/5/$ARCH/python-boto-1.9b-6.el5.noarch.rpm
 
−
 
 
−
rpm -Uvh duplicity-0.6.14-1.el5.$ARCH.rpm ncftp-3.2.2-1.el5.$ARCH.rpm librsync-0.9.7-13.el5.$ARCH.rpm python-GnuPGInterface-0.3.2-2.el5.noarch.rpm python-boto-1.9b-6.el5.noarch.rpm
 
−
</source>
 
  
 
[[Category: Installation Guide]]
 
[[Category: Installation Guide]]
 
[[Category: OKM Network]]
 
[[Category: OKM Network]]

Revision as of 11:29, 24 October 2011

These backup scripts use rsync to minimize network load and creates incremental backups, preserving last four backups. For more info, read http://www.mikerubel.org/computers/rsync_snapshots/.

To install the cron job, run: 

$ sudo crontab -e

And add these lines according to your personal configuration: 

MAILTO=nomail@openkm.com
@weekly /root/backup.sh | tee -a /root/backup.log

Or, if you want to separate log reports by date: 

MAILTO=nomail@openkm.com
@weekly /root/backup.sh | tee /root/backup.$(date +\%Y.\%m.\%d_\%H.\%M.\%S).log

Nota clasica.png MAILTO may also be used to direct mail to multiple recipients by separating recipient users with a comma.

For more information take a look at Crontab quick reference

Retrieved from "https://www.openkm.com/wiki/index.php?title=Backup_scripts&oldid=5379"