Difference between revisions of "Backup scripts"

From OpenKM Documentation
Jump to: navigation, search
(→‎Remote backup with rsync (rotation 2))
 
(55 intermediate revisions by 2 users not shown)
Line 1: Line 1:
−
These backup scripts use rsync to minimize network load and creates incremental backups, preserving last four backups. For more info, read http://www.mikerubel.org/computers/rsync_snapshots/.
+
{{TOCright}} __TOC__
 +
 
 +
These backup scripts use rsync to minimize network load and creates incremental backups, preserving last four backups. For more info, read http://www.mikerubel.org/computers/rsync_snapshots/. Also recommend the article [http://dar.linux.free.fr/doc/mini-howto/dar-differential-backup-mini-howto.es.html DAR differential backup mini-howto].
  
 
To install the cron job, run:
 
To install the cron job, run:
Line 19: Line 21:
 
</source>
 
</source>
  
−
== Remote backup with rsync ==
+
{{Note|MAILTO may also be used to direct mail to multiple recipients by separating recipient users with a comma.}}
−
<source lang="bash">
 
−
#!/bin/bash
 
−
#
 
−
## BEGIN CONFIG ##
 
−
HOST=$(uname -n)
 
−
FILES="/home/openkm"
 
−
## END CONFIG ##
 
−
echo -e "### BEGIN: $(date +"%x %X") ###\n"
 
  
−
# Stop JBoss
+
This script will export all databases from a MySQL installation:
−
/etc/init.d/jboss stop
 
−
while [ "$(ps -ef | grep java | grep jboss | wc -l)" -gt "0" ]; do
 
−
  sleep 5; echo ".";
 
−
done
 
  
−
# Copy to backup server
 
−
rsync -apzhR --stats --delete --exclude=*~ --delete-excluded $FILES backup@server:/home/backup/ $HOST/backup
 
−
 
−
# Start JBoss
 
−
/etc/init.d/jboss start
 
−
echo -e "\n### END: $(date +"%x %X") ###"
 
−
</source>
 
−
 
−
== Remote backup with rsync (rotation 1) ==
 
 
<source lang="bash">
 
<source lang="bash">
 
#!/bin/bash
 
#!/bin/bash
 
#
 
#
 
## BEGIN CONFIG ##
 
## BEGIN CONFIG ##
−
HOST=$(uname -n)
+
MYSQL_PASS=""
−
FILES="/home/openkm"
+
DATABASE_EXP="/home/openkm/db"
 
## END CONFIG ##
 
## END CONFIG ##
−
echo -e "### BEGIN: $(date +"%x %X") ###\n"
 
  
−
# Stop JBoss
+
rm -rf $DATABASE_EXP
−
/etc/init.d/jboss stop
+
mkdir -p $DATABASE_EXP
  
−
# Copy to backup server
+
# Backup de MySQL
−
ssh backup@server "cd $HOST; rm -rf backup.3; mv backup.2 backup.3; mv backup.1 backup.2; mv backup.0 backup.1"
+
if [ -n "$MYSQL_PASS" ]; then
−
rsync -apzhR --stats --delete --exclude=*~ --delete-excluded --link-dest="/home/backup/$HOST/backup.1" \
+
  MYSQL_DBS=$(mysqlshow -h localhost -u root -p$MYSQL_PASS | awk '(NR > 2) && (/[a-zA-Z0-9]+[ ]+[|]/) { print $2 }');
−
$FILES backup@server:/home/backup/ $HOST/backup.0
+
−
 
+
  for DB in $MYSQL_DBS; do
−
# Start JBoss
+
    if [[ $DB != "mysql" && $DB != "test" && $DB != "information_schema" && $DB != "performance_schema" ]]; then
−
/etc/init.d/jboss start
+
      echo "* Backuping MySQL data from $DB..."
−
echo -e "\n### END: $(date +"%x %X") ###"
+
      mysqldump -h localhost -u root -p$MYSQL_PASS $DB > $DATABASE_EXP/mysql_$DB.sql
−
</source>
+
    fi
−
 
+
  done
−
== Remote backup with rsync (rotation 2) ==
+
fi
−
<source lang="bash">
 
−
#!/bin/bash
 
−
#
 
−
## BEGIN CONFIG ##
 
−
HOST=$(uname -n)
 
−
FILES="/home/openkm"
 
−
BACKUP_DIR="/media/BACKUP"
 
−
## END CONFIG ##
 
−
echo -e "### BEGIN: $(date +"%x %X") ###\n"
 
−
 
 
−
# Stop JBoss
 
−
/etc/init.d/jboss stop
 
−
 
 
−
# Calculate snapshot
 
−
LAST_SNAPSHOT=`ls -ltr $BACKUP_DIR | tail -1 | awk {'print $8'} | cut -d . -f 2`
 
−
NEW_SNAPSHOT=$((LAST_SNAPSHOT+1))
 
−
 
 
−
# Copy to backup server
 
−
rsync -apzhR --stats --delete --exclude=*~ --delete-excluded \
 
−
--link-dest="$BACKUP_DIR/backup.$LAST_SNAPSHOT" $FILES "$BACKUP_DIR/backup.$NEW_SNAPSHOT"
 
−
 
 
−
# Start JBoss
 
−
/etc/init.d/jboss start
 
−
echo -e "\n### END: $(date +"%x %X") ###"
 
 
</source>
 
</source>
  
−
== Remote backup with rdiff-backup ==
+
This script will export all databases from a PostgreSQL installation:
−
<source lang="bash">
 
−
#!/bin/bash
 
−
#
 
−
## BEGIN CONFIG ##
 
−
HOST=$(uname -n)
 
−
FILES="/home/openkm"
 
−
## END CONFIG ##
 
−
echo -e "### BEGIN: $(date +"%x %X") ###\n"
 
−
 
 
−
# Stop JBoss
 
−
/etc/init.d/jboss stop
 
−
 
 
−
# Rotate and copy files
 
−
rdiff-backup --remove-older-than 30B backup@server:/home/backup/$HOST
 
−
rdiff-backup -v 3 --print-statistics --exclude /media \
 
−
  --include /etc --include /root --include /home \
 
−
  --exclude '**' / backup@server:/home/backup/$HOST
 
−
 
 
−
# Start JBoss
 
−
/etc/init.d/jboss start
 
−
echo -e "\n### END: $(date +"%x %X") ###"
 
−
</source>
 
  
−
== Local backup to USB disk ==
 
 
<source lang="bash">
 
<source lang="bash">
 
#!/bin/bash
 
#!/bin/bash
 
#
 
#
 
## BEGIN CONFIG ##
 
## BEGIN CONFIG ##
−
MYSQL_PASS=""
 
−
JBOSS_HOME="/home/openkm/jboss-4.2.3.GA"
 
 
DATABASE_EXP="/home/openkm/db"
 
DATABASE_EXP="/home/openkm/db"
−
BACKUP_DIR="/media/Backup"
 
−
FILES="$JBOSS_HOME $DATABASE_EXP"
 
 
## END CONFIG ##
 
## END CONFIG ##
  
−
if [ $(id -u) != 0 ]; then echo "You should run this script as root"; exit; fi
 
−
 
−
# Mount disk
 
−
mount $BACKUP_DIR
 
−
 
−
echo -e "### BEGIN: $(date +"%x %X") ###\n"
 
 
rm -rf $DATABASE_EXP
 
rm -rf $DATABASE_EXP
 
mkdir -p $DATABASE_EXP
 
mkdir -p $DATABASE_EXP
  
−
# Stop JBoss
+
# Backup de PostgreSQL
−
/etc/init.d/jboss stop
+
POSTGRESQL_DBS=$(su postgres -c "psql -l" | awk '(NR > 2) && (/[a-zA-Z0-9]+[ ]+[|]/) { print $1 }');
  
−
while [ "$(ps -ef | grep java | grep jboss | wc -l)" -gt "0" ]; do
+
for DB in $POSTGRESQL_DBS ; do
−
  sleep 5; echo ".";
+
  if [[ $DB != "template0" && $DB != "template1" && $DB != "postgres" ]]; then
 +
    echo "* Backuping PostgreSQL data from $DB..."
 +
    su postgres -c "pg_dump -Fc -b $DB" > $DATABASE_EXP/pg_$DB.dmp
 +
  fi
 
done
 
done
 +
</source>
  
−
# Clean logs
+
For more information take a look at [http://adminschoice.com/crontab-quick-reference Crontab quick reference]
−
echo "Clean JBoss temporal files."
 
−
rm -rf $JBOSS_HOME/server/default/log
 
−
rm -rf $JBOSS_HOME/server/default/tmp
 
−
rm -rf $JBOSS_HOME/server/default/work
 
  
−
# Backup de MySQL
+
* [[Backup with LFTP]]
−
if [ -n "$MYSQL_PASS" ]; then
+
* [[Backup with rsync]]
−
  MYSQL_DBS=$(mysqlshow -h localhost -u root -p$MYSQL_PASS | awk '(NR > 2) && (/[a-zA-Z0-9]+[ ]+[|]/) && ($2 != "mysql") && ($2 != "test") && ($2 != "information_schema") { print $2 }');
+
* [[Backup with rdiff-backup]]
−
 
+
* [[Backup with duplicity]]
−
  for DB in $MYSQL_DBS ; do
 
−
    echo "* Backuping MySQL data from $DB...";
 
−
    mysqldump -h localhost -u root -p$MYSQL_PASS $DB > $DATABASE_EXP/mysql_$DB.sql
 
−
  done
 
−
  echo "-------------------------------------";
 
−
fi
 
−
 
 
−
# Rotate and copy files
 
−
cd $BACKUP_DIR; rm -rf mv backup.3 backup.4; mv backup.2 backup.3; mv backup.1 backup.2; mv backup.0 backup.1; cd
 
−
rsync -apzhR --stats --delete --exclude=*~ --exclude="$JBOSS_HOME/cache" --delete-excluded --link-dest="$BACKUP_DIR/backup.1" $FILES "$BACKUP_DIR/backup.0"
 
−
 
 
−
# Start JBoss
 
−
/etc/init.d/jboss start
 
−
echo -e "\n### END: $(date +"%x %X") ###"
 
−
 
 
−
# Status
 
−
echo "=================================";
 
−
du -hs $BACKUP_DIR
 
−
echo "*********************************";
 
−
du -hs --time $BACKUP_DIR/*
 
−
echo "=================================";
 
−
df -h | grep "$BACKUP_DIR"
 
−
echo "=================================";
 
−
 
 
−
# Umount disk
 
−
sync
 
−
umount $BACKUP_DIR
 
−
</source>
 
  
 
[[Category: Installation Guide]]
 
[[Category: Installation Guide]]
−
[[Category: OKM Network]]
 

Latest revision as of 14:01, 10 March 2014

These backup scripts use rsync to minimize network load and creates incremental backups, preserving last four backups. For more info, read http://www.mikerubel.org/computers/rsync_snapshots/. Also recommend the article DAR differential backup mini-howto.

To install the cron job, run: 

$ sudo crontab -e

And add these lines according to your personal configuration: 

MAILTO=nomail@openkm.com
@weekly /root/backup.sh | tee -a /root/backup.log

Or, if you want to separate log reports by date: 

MAILTO=nomail@openkm.com
@weekly /root/backup.sh | tee /root/backup.$(date +\%Y.\%m.\%d_\%H.\%M.\%S).log

Nota clasica.png MAILTO may also be used to direct mail to multiple recipients by separating recipient users with a comma.

This script will export all databases from a MySQL installation: 

#!/bin/bash
#
## BEGIN CONFIG ##
MYSQL_PASS=""
DATABASE_EXP="/home/openkm/db"
## END CONFIG ##

rm -rf $DATABASE_EXP
mkdir -p $DATABASE_EXP

# Backup de MySQL
if [ -n "$MYSQL_PASS" ]; then
  MYSQL_DBS=$(mysqlshow -h localhost -u root -p$MYSQL_PASS | awk '(NR > 2) && (/[a-zA-Z0-9]+[ ]+[|]/) { print $2 }');
 
  for DB in $MYSQL_DBS; do
    if [[ $DB != "mysql" && $DB != "test" && $DB != "information_schema" && $DB != "performance_schema" ]]; then
      echo "* Backuping MySQL data from $DB..."
      mysqldump -h localhost -u root -p$MYSQL_PASS $DB > $DATABASE_EXP/mysql_$DB.sql
    fi
  done
fi

This script will export all databases from a PostgreSQL installation: 

#!/bin/bash
#
## BEGIN CONFIG ##
DATABASE_EXP="/home/openkm/db"
## END CONFIG ##

rm -rf $DATABASE_EXP
mkdir -p $DATABASE_EXP

# Backup de PostgreSQL
POSTGRESQL_DBS=$(su postgres -c "psql -l" | awk '(NR > 2) && (/[a-zA-Z0-9]+[ ]+[|]/) { print $1 }');

for DB in $POSTGRESQL_DBS ; do
  if [[ $DB != "template0" && $DB != "template1" && $DB != "postgres" ]]; then
    echo "* Backuping PostgreSQL data from $DB..."
    su postgres -c "pg_dump -Fc -b $DB" > $DATABASE_EXP/pg_$DB.dmp
  fi
done

For more information take a look at Crontab quick reference

Retrieved from "https://www.openkm.com/wiki/index.php?title=Backup_scripts&oldid=11336"