Backup setup

From Smith family
Jump to: navigation, search
Server setup
← Previous Next →
Log stats Discourse

I have a simple process for having my two machines back up themselves every night. This is kicked off with a cron script. Personally, I do full backups every time (rather than incremental backups) as it makes recovery easier.


Create users

Backups are safer if they use non-privileged users to do the backups. For instance, create the backup user in MySQL. This user has read-only access to all databases, and does not require a password. Create the user by entering the following commands:

root@desktop:~# mysql -u 'root' -p 
mysql> grant select, lock tables on *.* to 'backup'@'localhost';
mysql> quit;

This automatically creates the user, with no password, and gives them permissions to read all MySQL databases.


For Postgres, it's a bit more complicated. Create the user, connect to the database to back up, and grant permissions.

root@desktop:~# sudo -u postgres psql
postgres=# create user backup with login encrypted password 'mypassword';
CREATE ROLE
postgres=# \c dmarc
psql (9.3.13, server 9.1.18)
You are now connected to database "dmarc" as user "postgres".
postgres=# grant connect on database dmarc to backup;
GRANT
postgres=# grant select on all tables in schema public to backup;
GRANT
postgres=# grant select on all sequences in schema public to backup;
GRANT

Then create the file /root/.pgpass with contents

# hostname:port:database:username:password
*:*:*:backup:mypassword
(giving the password you've used) and give it the correct permissions:
root@desktop:~# chmod 0600 ~/.pgpass

Desktop backup

On the desktop machine, there is /etc/cron.daily/makedailybackups:

#!/bin/bash

BACKUPFILENAME="/backup/daily-backup.desktop.$(date -I)"

# Create new backup files.  Do them in subshells so they get done in parallel
(tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \
    chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) &
# (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \
#     chmod 600 "$BACKUPFILENAME.music.tar.bz2") &
# (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \
#     chmod 600 "$BACKUPFILENAME.sources.tar.bz2") &
(tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \
    chmod 600 "$BACKUPFILENAME.pictures.tar.bz2") &
(tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \
    chmod 600 "$BACKUPFILENAME.etc.tar.bz2") &
(tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \
    chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & 

wait

# Remove backups more than 2 days old
find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -mtime +1 -exec rm -f {} \; 

and /etc/cron.monthly/makemonthlybackups

#!/bin/bash

BACKUPFILENAME="/backup/monthly-backup.desktop.$(date -I)" 

# Create new backup files.  Do them in subshells so they get done in parallel
(tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \
    chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) &
(tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \
    chmod 600 "$BACKUPFILENAME.music.tar.bz2") &
(tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \
    chmod 600 "$BACKUPFILENAME.sources.tar.bz2") &
(tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \
    chmod 600 "$BACKUPFILENAME.photos.tar.bz2") &
(tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \
    chmod 600 "$BACKUPFILENAME.etc.tar.bz2") &
(tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \
    chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & 

wait

# Remove backups more than 40 days old
find /backup/ -maxdepth 1 -name 'monthly-backup*' -type f  -mtime +40 -exec rm -f {} \;

In either case, a file is restored with the command

root@desktop:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx

Adding the option --absolute-names would restore the files to paths anchored at root.

To extract specific files, use

root@desktop:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx --wildcards --no-anchored 'make*backup'

which will extract files that match the given pattern (--wildcards), regardless of their preceeding path name (--no-anchored).

Server backup

Backups on the server are a little more complicated, because of all the various services the server provides. Here's /etc/cron.daily/makedailybackups, the daily backup script:

#!/bin/bash

BACKUPFILENAME="/backup/daily-backup.ogedei.$(date -I)"

# Remove backups more than 4 days old                                                                                                                              
find /backup/         -maxdepth 1 -name 'daily-backup*' -type f -daystart -mtime +5 -exec rm -f {} \;

# Create file backups                                                                                                                                              
nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.opt.tar.bz2" \
        --exclude='/opt/svn/*' /opt
nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc
nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \
        --exclude='/home/git/*' /home
nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.vmail.tar.bz2" /var/vmail
nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.git.tar.bz2" /home/git
nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.www.tar.bz2" /var/www

# Create database backups                                                                                                                                          
for db in njaewikidb mkrpgwikidb ; do
    nice mysqldump --user='backup' --opt --databases $db | nice bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2"
    chmod 400 "${BACKUPFILENAME}.${db}-dump.sql.bz2"
done

pg_dump -U backup -Fc dmarc > "${BACKUPFILENAME}.dmarc-dump"
chmod 400 "${BACKUPFILENAME}.dmarc-dump"
 
for f in "$BACKUPFILENAME.opt.tar.bz2" "$BACKUPFILENAME.etc.tar.bz2" "$BACKUPFILENAME.home.tar.bz2" \
    "$BACKUPFILENAME.vmail.tar.bz2" "$BACKUPFILENAME.git.tar.bz2" "$BACKUPFILENAME.www.tar.bz2" \
    "$BACKUPFILENAME.njaewikidb-dump.sql.bz2" "$BACKUPFILENAME.mkrpgwikidb-dump.sql.bz2" \
    "$BACKUPFILENAME.dmarc-dump"
do
    chmod 400 $f
    nice scp $f user@desktop.domain.tld:/backup
    ssh user@desktop.domain.tld "/backup/lockdown $f"
done

To restore MySQL dumps, you may need to create the database in MySQL:

root@server:~# mysql -u 'root'@'localhost' -p -e 'create database DB_NAME'

Then get MySQL to run it:

root@server:~# bunzip2 DB_DUMP_FILE | mysql --user='root' -p

To restore Postgres dumps, drop the database then restore from the backup dump:

root@server:~# sudo -u postgres dropdb DATABASE                                                                                                                                 
root@server:~# sudo -u postgres pg_restore -Fc -C -d postgres DB_DUMP_FILE

Cron (and Anacron) timing

I found that the default timing for kicking off cron jobs (at around 7.30) meant that the backups were still being created when I started work. That was easily fixed by altering /etc/crontab on the server so that the hours column (the second column) is 1 instead of 7:

25 1    * * *   root    test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily )
47 1    * * 7   root    test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.weekly )
52 1    1 * *   root    test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.monthly )

On the desktop machines, alter /etc/cron.d/anacron to contain:

30 1    * * *   root    test -x /etc/init.d/anacron && /usr/sbin/invoke-rc.d anacron start >/dev/null

If needed, edit the contents of the files in /var/spool/anacron/ to make weekly and monthly cron jobs happen on sensible days. Use touch to adjust the datestamps to the same dates.

root@desktop:~# touch -m -t 201005010100 /var/spool/anacron/cron.monthly