Backup SME Server Remotely Using cURL

From SME Server
Jump to navigation Jump to search

Introduction

This HowTo describes a simple way to backup your SME Server remotely using cURL and requires no changes to your SME Server. Further, cURL requires no user interaction and can be scripted or automated using cron to perform regular backups. If the file system the backup is sent to is large file enabled (not DOS) this is also a fairly reliable way to get a desktop style backup > 4GB.

The following is updated, tested and confirmed for SME Server 9

Does not work with latest 9.1 updates. A fix is needed to in the core SME admin pages to make this work again. This has been filed as BUG 10166 in bugzilla

Pre-Requisites

There are two simple pre-requisites.

  1. The machine performing the backup must have web access to the server-manager of the SME Server being backed up. That is, you must be able to get to the manual web backup page from a web browser on the machine that will do the backup.
  2. The machine that will perform the backup must have a copy of curl installed on it.

The Basics

The basic concept is very simple. We will use the curl command to imitate the manual desktop backup procedure. During a manual desktop backup you do two things:

  1. Log on to the Server Manager.
  2. Perform a "Backup to Desktop" from the Backup and restore page.

These two actions map to the following cURL commands: 

curl -s -k -b ~/.sme_cookies -c ~/.sme_cookies -F username='admin' -F password='your_password' -F back='https:/your_sme_server/' \
        https://your_sme_server/server-common/cgi-bin/login

curl -f -s -k -b ~/.sme_cookies -H 'Expect:' -F function=desktop-backup -F state=perform \
        https://your_sme_server/server-manager/cgi-bin/backup > backupfile.tar.gz

Where:

  • Each curl commands should be on one line and the \ removed
  • your_password is replaced by your password
  • your_sme_server is replaced with the name or IP number of the SME Server you wish to backup
  • backupfile.tar.gz is the name you want to save the backup as


  Warning:
If you are sharing the machine doing the backup with others using -F "password=your_password" is not secure. Any user can see your password with ps if they can catch it while the command is running!

cURL does however allow you to save your password in a file and read it in as follows: 

echo your_password >~/sme_password
Then you can replace "password=your_password" in the above with "password=<~/sme_password"


  Note:
Full information on cURL can be found in the cURL manual

The options I used are:

  • -f Fail silently
  • -s Silent, I.E. don't show a progress meter
  • -k Don't insist on SSL certs signed by trusted CA's (most SME Servers use self signed certificates)
  • -b <cookies> Send cookies to server
  • -c <cookies> Save cookies here, this is where it stashes your login credentials between step 1) and 2)
  • -H Used to set the HTML Expect: header to blank (was not required prior to SME Server 9)
  • -F Fill in form field
  • <url>

While this process works and illustrates the principle it does raise some security concerns and can be improved. (see below)


Scripting cURL and Improving Security

Now that we have shown that we can pull a backup from an SME Server without any user interaction and without adding any software to the standard SME Server installation we can now wrap the whole process up in a simple script.

As the machine I was using to do the backup on was also a Linux machine, I have used sh but if your machine is Windows you could do something similar in a .bat file.


  Note:
All the following is done on the machine doing the backup NOT the SME Server you are backing up. No changes are ever made to that SME Server.


On my backup machine I have a user smebackup (any user can be used) and in its home folder I have a bin folder to contain the do_backups script. This script contains some simple enhancements to the above cURL commands: 

#!/bin/sh

curl -s -k -b ~/.sme_cookies -c ~/.sme_cookies -F username=admin -F "password=<~/sme_password" -F back='https://snoopy/' \
         <nowiki>https://snoopy/server-common/cgi-bin/login && \
curl -f -s -k -b ~/.sme_cookies -H 'Expect:' -F function=desktop-backup -F state=perform \
        https://snoopy/server-manager/cgi-bin/backup > /home/smebackup/sme-snoopy-`/bin/date +\%d`.tgz

In this script we have combined the two curl commands via an && so that the second command will only run if the first command succeeds.

The use of date in the filename will append the day of the month to the filename. This will mean that a new backup will be created for every day of the month and overwrite them next month, obviously if the next month has fewer days you may be left with a couple of backups from last month. This is for use in automated backups (see below). This process can be further enhanced or changed as required.

This script has also been enhanced to read the password from the file sme_password also located in the users home directory. You will need to create the sme_password file. It should simply contain one line that is the password of your SME Server you wish to backup


  Warning:
There are a couple of security considerations that you should consider when using curl.
  1. Using "password=your_password" in cURL will cause the password to appear momentarily in the ps listing. As described above, this is best avoided by placing the password in a file and using -F "password=<your_password_file" .
  2. The cookie jar ~/.sme_cookies should not be made world readable otherwise others could use your downloaded cookies. Likewise the password file should also be protected!


Automating backups with cron

Automating the backups is now simply a matter of creating a cron job to perform the backup using crontab -e. My smebackup user's cron entry is: 

45 21 * * * umask 077 && /home/smebackup/bin/do_backups

This starts a backup at 21:45 every day. The umask prevents other users on the machine reading the backups.

Restoring your backup

In versions of SME Server prior to 7.4 it was possible to restore the backup through the web interface. This was removed in 7.4 making it very difficult to manage a restore remotely. The recommended official restore procedure is to build a whole new machine. Some alternative ways of restoring a backup are described in Backup server config.

Other Extensions

In my case several of the machines I backup are behind firewalls and can not be accessed via ssh or alternative methods. This however is not a problem for cURL if the firewalls allow http/https traffic even if a proxy needs to be used.

To use a proxy with cURL you can simply add: 

-x your_proxy_server:your_proxy_port

For Example: 

curl -s -k -x stargate:3128 -b ~/.sme_cookies -c ~/.sme_cookies -F username=admin \
        -F "password=<~/sme_password" https://snoopy/server-common/cgi-bin/login && \
curl -f -s -k -x stargate:3128 -b ~/.sme_cookies -F function=desktop-backup -H "Expect:" -F state=perform \
        https://snoopy/server-manager/cgi-bin/backup > /home/smebackup/sme-snoopy-`/bin/date +\%d`.tgz
Retrieved from "https://wiki.koozali.org/index.php?title=Backup_SME_Server_Remotely_Using_cURL&oldid=33215"