Simple Backup Scheme - Lookaway CMS

Documented by Kyle Bruder on Mar 22, 2021
Last updated on Mar 23, 2021


Untitled_Diagram2

Introduction

Learn how to automatically backup your Lookaway media and database once per day and remove backups older than one week. Data can be lost in so many ways for so many reasons. The most common cause is human error. The way to mitigate the risk of permanent data loss is to create redundant copies of the data at regular intervals.

In this example, we are using Ubuntu 20 and PostgreSQL. With slight modifications, these commands will work on most Linux kernels and RDMS (relation database management systems). The users, file paths and database names used in the example code are intended to be consistent with other examples on the site in order to avoid the use of meta syntax.


  1. Check Disk Space

    Before creating new data, be sure you have enough space on your storage device. Filling a disk over 80% capacity will cause performance loss.

    $ df -h
    Filesystem                Size  Used Avail Use% Mounted on
    ...
    /dev/mapper/raid0         984G  7.7G  926G   1% /home/lookaway/lookaway-env/lookaway/media
    /dev/mapper/backups       2.0T   60G  1.8T   4% /home/lookaway/backups
    ...
    $ du -sh ~lookaway/lookaway-env/lookaway/media
    7.7G	/home/lookaway/lookaway-env/lookaway/media
    

  2. Backup Your Lookaway Database

    Use the following command to "dump" the Lookaway database to a file. It is generally a good idea to do this before backing up the media. Although both the database and media directory will shrink and grow over time and objects are created and deleted, the trend is toward growth. It is better to have a younger media backup because if the media directory is modified after the database is backed up, then "worst case" scenario you will have one or more an "orphaned" files. If the media is backed up first and the database changes, objects may point to non existent files if they were uploaded between the time the backup of the media started and the database finishes.

    Tip: Make sure you use this command with an account that has permission to read the database.

    $ pg_dump lookaway > ~/backups/test/lookaway_db.sql
    

    Dump your Lookaway Database into a file.
    Source: kbruder Tech

  3. Backup Your Lookaway Media Files

    Create a compressed tape archive of your project's media directory using the 'tar' command.

    Tip: If you aren't already there, change your working directory to the base directory ('/home/lookaway/lookaway-env/lookaway/' in this example) of the Lookaway or use the absolute path to your project's media directory.

    tar cvfpz ~/backups/test/lookaway_media.tar.gz media
    

    Use this line to backup your Lookaway media files.
    Source: kbruder Tech

  4. Create a Backup Script

    Open a new file and create a simple script. This script will create a timestamped directory where your Lookaway database and media files will be copied. Each directory will have only two files; a "dump" of the database and a "tape archive" of the media directory. As long as the two backup files were created form the same data state (around almost exactly at the same time), you will be able to restore the set of backup data to a fresh installation on a new server or your existing server. If the backup data was taken from an older version of the Lookaway code that used a different database schema, then you will need to pull the that version of the master branch.

    Tip: Make sure you have permission to write new files to the path you use in the script.

    Caution: Running this script more than once on a given calendar day will override previous backups made on that day.

    ~/lookaway_backup.sh
    # Set variables according to your specs.
    backup_dir="/home/lookaway/backups" # Your desired backup directory
    lookaway_media_path="/home/lookaway/lookaway-env/lookaway/media" # The path to the lookaway media files
    database="lookaway"
    
    # Create today's backup dir.
    today=$(date +%y-%m-%d)
    if [ ! -d $backup_dir/$today ]; then
    	echo "$(date) - Creating Lookaway backup dir $backup_dir/$today"
    	/usr/bin/mkdir $backup_dir/$today
    fi
    
    # Dump the Database
    /usr/bin/pg_dump lookaway > $backup_dir/$today/lookaway_db.sql 
    
    # Create the archive
    /usr/bin/tar cvfpz $backup_dir/$today/lookaway_media.tar.gz $lookaway_media_path
    

    Creates a compressed tape archive of the Lookaway media directory (images, sounds, video, etc.) and the Lookaway database in a timestamped folder.
    Source: kbruder Tech

  5. Modify the Script Permissions

    Set the execution bit on the script file

    $ chmod u+x ~/lookaway_backup.sh
    

    Allows the user (owner of the file) to execute a file.
    Source: kbruder Tech

  6. Add Cron Jobs

    Edit your user's crontab and add the backup script. The first line runs the backup script at 5:00 AM everyday. The next line removes files older than 7 days in the '/home/lookaway/backups' directory once per day at 6:00 AM. The final line removes directories older than 7 days in the '/home/lookaway/backups' directory once per day at 6:30 AM.

    Tip: All times are according to the system clock (usually UTC) which may be different from your local timezone.

    Caution: Avoid the use of relative paths when adding crontab lines, especially with commands. When the script is executed by a cron job, the $PATH environment variable may be different than the one sourced by your terminal shell.

    crontab -e
    

    # m h  dom mon dow   command
    
    # Lookaway Backup Script
    0 5 * * * /home/lookaway/lookaway_backup.sh
    
    # Remove backups older than 7 days
    0 6 * * * /bin/find /home/lookaway/backups -type f -mtime +7 -exec /bin/rm  {} \;
    30 6 * * * /bin/find /home/lookaway/backups -type d -mtime +7 -exec /bin/rmdir  {} \;
    

    The first line runs a backup script at 5:00 AM everyday. The next line removes files older than 7 days in the '/home/lookaway/backups' directory. The final line removes directories older than 7 days in the '/home/lookaway/backups' directory.
    Source: kbruder Tech

  7. Ensure Your Script and Cron Jobs Work as Expected

    Check to make sure that your backups are being created and removed as expected. It is unacceptable to assume that everything is working. After a day or two list the contents of your backup directory. You should see the newly created timestamped directories there. The modification times of the directories and files should closely reflect the time set on the crontab. If you discover unexpected results or no files at all, then check the system log and scroll to the lines that match the time specified in the crontab in order to find clues as to where the failure(s) is/are occurring.

    Tip: Once in the 'less' file viewer, use the '/' key to search the log. Search using the string 'CRON', the username, the script filename, or the timestamp in order to quickly find the relevant lines.

    Caution: You need sudo permissions to read the system log.

    /var/log/syslog
    $ sudo less /var/log/syslog
    ...
    Mar 22 05:00:02 goodship1 CRON[69899]: (lookaway) CMD (/home/lookaway/lookaway_backup.sh)
    ...
    

    Use the '/' key to search the log. Search using the string 'CRON', the username, the script filename, or the timestamp in order to quickly find the relevant lines.
    Source: kbruder Tech

    $ ls -la ~/backups/21-03-15
    ...
    -rw-rw-r--  1 lookaway lookaway     851128 Mar 15 05:00 lookaway_db.sql
    -rw-rw-r--  1 lookaway lookaway 7999989274 Mar 15 05:03 lookaway_media.tar.gz
    

    Show a more detailed listing of a file or directory contents including the permissions, user, group size and modifcation time.
    Source: kbruder Tech

Conclusion

This is the most basic way to keep regular backups of your data. Backups will help you in the event your data is lost. Regular backups are great for disaster recovery. When making changes to a production environment, make backups right before the changes are deployed so you have the most up to date state of your data. Backup data can be used to migrate data to new database and files storage systems. Be sure to implement redundancy in your system design, e.g. RAID, SAN, distributed block device, remote sites, cloud storage, etc..


Referenced by

Recovering From Backup Data - Lookaway CMS

This document explains how to recover your Lookaway CMS powered website from backup data.

bitcoin:3MnhNRKgrpTFQWstYicjF6GebY7u7dap4u
Bitcoin Accepted Here

3MnhNRKgrpTFQWstYicjF6GebY7u7dap4u

Please donate some Bitcoin to our site. We can use it to keep improving the site and open up to more members. Any amount will help. Thank you.


litecoin:MT61gm6pdp9rJoLyMWW5A1hnUpxAERnxqg
Litecoin Accepted Here

MT61gm6pdp9rJoLyMWW5A1hnUpxAERnxqg

Please donate some Lite to our site. We can use it to keep improving the site and open it up to more members. Any amount will help. Thank you.