Difference between revisions of "Backup server config"
m (add note) |
|||
| Line 1: | Line 1: | ||
| + | {{usefulnote}} | ||
== Backup server configuration == | == Backup server configuration == | ||
Revision as of 19:12, 16 October 2014
Is this article helpful to you?
Please consider donating or volunteering
Thank you!
Backup server configuration
Overview
This method is a good way to create a server clone that has identical configuration but no data, without going through all the setup again.
This method utilizes the standard backup to desktop function and the restore from CD or USB disk at first startup function.
The non standard part of this procedure is that data in ibays and user folders and other specific locations, is temporarily moved to a newly created /temp folder, which will not be included in the backup (as /temp is not in the standard list of backup inclusions).
The main reason for doing this is to keep the resultant smeserver.tgz backup file size smaller than 2Gb, so that there are no problems doing a backup to desktop on a workstation using the http protocol and any browser.
If you also wish to transfer data, then user, ibay and other data in other specific locations can be manually copied from the backup server to the restored server either partially or in total after the restore. Use tar and scp to transfer this data so that correct permissions and ownership are retained, see the section "Copying data to the restored server".
If you have large amounts of data in other locations that will cause your backup file to exceed 2Gb, then that data should also be temporarily moved to the /temp folder, and manually copied to the new server after the restore.
Procedure
- On the server to be backed up (server1) do mkdir -p /temp mv /home/e-smith/files/users /temp mv /home/e-smith/files/ibays /temp mv /home/e-smith/files/xxxx /temp
- Perform backup to desktop on server1 using the standard server manager panel
- Then on server1 do mv /temp/users /home/e-smith/files/ mv /temp/ibays /home/e-smith/files/ mv /temp/xxxx /home/e-smith/files/
- Build another server to restore to (server2), and perform a fresh install of sme server on it. Note that any preferred hardware can be used, it does not need to be physically identical to the original server the backup was performed on.
- Copy the smeserver.tgz backup file to a CD or attached USB disk. When you first start server2 after installing the fresh OS, you will be asked if you want to do a Restore. Select the appropriate source ie CD or USB. Note you only see this Restore option the first time your start the server2, so do not skip using this option as it will not be presented on future restarts.
- After the restore is complete on server2, login as admin and select Configure this server. Change the local IP and other settings as required, detect different NIC's, and also change to server only mode if it's going to be a second server on your network (where the backed up server was a server/gateway) and disable the DHCP server. As a minimum, make sure you change the local IP on the restored server as it will be identically configured after the restore. Then log in to server manager on the restored server and change workgroup name and domain controller settings and any other settings required etc.
You should now have a second server (server2) which is identically configured to server1 (except for changes that were made) but without user and ibay data. Note that the user and ibays will exist in server manager and in the file structure, but they will be empty.
Copying data to the restored server
You can then optionally transfer the data across the network from server to server either totally or in limited form as required using command line, tar and scp. You can delete any users, ibays, groups and other settings not required on the restored server via server manager.
It is preferable to tar/untar data so you don't loose ownership and permissions during the copy from server1 to server2. If copying user data and email using normal copy commands, then ensure that ownership is correct on the new server.
Modify the following commands appropriately to only move the data you require.
- Use tar to compress all the user and ibays folders (on server1) by doing cd /home/e-smith/files tar -czvf /temp/users.tar.gz users tar -czvf /temp/ibays.tar.gz ibays tar -czvf /temp/xxxx.tar.gz xxxx
- Use scp to copy the tarred files between machines by doing (on server1)
scp /temp/users.tar.gz server2IP:/home/e-smith/files/ scp /temp/ibays.tar.gz server2IP:/home/e-smith/files/ scp /temp/xxxx.tar.gz server2IP:/home/e-smith/files/
If ssh is configured for a different port (than 22) use
scp -P zzzz /temp/users.tar.gz server2IP:/home/e-smith/files/ scp -P zzzz /temp/ibays.tar.gz server2IP:/home/e-smith/files/ scp -P zzzz /temp/xxxx.tar.gz server2IP:/home/e-smith/files/
where zzzz = port number
- Use tar to uncompress all the user and ibay folders by doing (on server2) cd /home/e-smith/files/ tar -xvzf users.tar.gz tar -xvzf ibays.tar.gz tar -xvzf xxxx.tar.gz rm /home/e-smith/files/users.tar.gz rm /home/e-smith/files/ibays.tar.gz rm /home/e-smith/files/xxxx.tar.gz If required, to correct ownership for each user on server2 do cd /home/e-smith/files/users chown -R username:username username
Manually transferring configuration information
If you wish to manually transfer configuration information from an old server to a new server you can use the following steps. All the files listed below are required as user information & passwords is contained in many different locations.
On the old server do the following to dump the mysql tables to a file in /home/e-smith/db/mysql/:
signal-event pre-backup
Before restoring files to the new server, issue the following command on the new server:
signal-event pre-restore
Then copy all the following files & folders from the old server to the new server preferably using the tar & scp commands, see usage examples & syntax above in the section "Copying data to the restored server".
You can exclude ibay, user & other data if preferred by moving that data to /temp (see steps in above section "Procedure")
This list is based on the requirements of sme7.x:
/etc/e-smith/templates-custom /etc/e-smith/templates-user-custom /etc/group /etc/gshadow /etc/passwd /etc/samba/secrets.tdb /etc/samba/smbpasswd /etc/shadow /etc/smbpasswd /etc/ssh /etc/sudoers /home/e-smith /root
and add in
/opt
if you have stuff installed there.
After copying all the required files to the new server you then need to run the following commands on the new server:
signal-event post-upgrade reboot
Cloning a server for Test purposes, using inclusion & exclusion files
This alternative method can be used to clone a server for test purposes, and utilises inclusion & exclusion text files to specify what the backup tgz file will contain. Refer forum post http://forums.contribs.org/index.php/topic,46546.msg228428.html#msg228428
Warning: This procedure is meant to be used for testing purposes ONLY, where you wish to trial changes etc before implementing them on a "near to identically configured" production server.
Procedure to create the backup on the server to be cloned
Create a file named include.txt in /root/ with this content:
touch /root/include.txt pico -w /root/include.txt
Edit the file to contain the following (which is a deviation from the standard backup inclusion set of files & folders)
home/e-smith/ etc/e-smith/templates-custom etc/e-smith/templates-user-custom etc/group etc/gshadow etc/passwd etc/samba/secrets.tdb etc/samba/smbpasswd etc/shadow etc/smbpasswd etc/ssh etc/sudoers root
Save & exit
Ctrl o Ctrl x
Create a file named /root/exclude.txt with this content:
touch /root/exclude.txt pico -w /root/exclude.txt
Edit the file to contain the following (which is the list of files & folders you wish to exclude from the backup)
home/e-smith/files/ibays/*/files/* home/e-smith/files/users/*/home/*
Save & exit
Ctrl o Ctrl x
Run the pre-backup event (to dump the mysql tables to a file in /home/e-smith/db/mysql/)
signal-event pre-backup
Run the tar command to create the smeserver.tgz backup file (the --totals directive is not strictly necessary, it is to provide information only)
tar -cz --totals -vvp --directory / --exclude-from=./exclude.txt --files-from=./include.txt -f /tmp/smeserver.tgz
Procedure to restore the backup on the cloned server
Copy the smeserver.tgz file to the cloned server via ssh or via usb disk and follow the steps in this section of the Backup server config wiki article http://wiki.contribs.org/Backup_server_config#Command_line_manual_Restore_from_USB_drive_or_memory_stick
Ensure you prepare the new server for the restore, by issuing the following command
signal-event pre-restore
and then follow the steps referred to above.
Performing the Procedures indicated above, will create a "near to identical" clone of the original server without user data and files in ibays. It will still contain user mail messages.
Standard backup & restore inclusions
This command will give a listing of the files & folders included in a standard backup & restore, as per a default install of sme server.
perl -e 'use esmith::Backup; $b = new esmith::Backup;print join("\n",$b->restore_list)'
Which on a sme7.5.1 server shows:
home/e-smith etc/e-smith/templates-custom etc/e-smith/templates-user-custom etc/ssh root etc/sudoers etc/passwd etc/shadow etc/group etc/gshadow etc/samba/secrets.tdb etc/samba/smbpasswd etc/smbpasswd
All files/scripts associated with the standard backup and restore processes can be found with this command.
rpm -ql e-smith-backup
The inclusion list can be found in
/usr/lib/perl5/site_perl/esmith/Backup.pm
Changes to this file will be overwriten during upgrades.
Modifying the standard backup inclusions & exclusions
To modify the included & excluded folders & files, refer to http://wiki.contribs.org/Backup_with_dar#Adding.2FExcluding_Directories_and_Files_from_the_backup_list
Data size calculation for backups
Depending on where data is stored on your server, the backup size reported by the server-manager may be less accurate eg if a lot of data is stored in /opt
A simple command can be saved as a script file and run when required to calculate the data size on the server for the folders backed up by default. Additional folders eg /opt can be included to suit particular requirements. The calculated size can be considered the largest possible backup size, as it is the size before any compression is applied. The degree of compression depends on content/file type, so the actual resultant backup file size will vary from server to server. The default backup folder list is obtained from /usr/lib/perl5/site_perl/esmith/Backup.pm
mkdir -p /opt/scripts cd /opt/scripts pico -w datasize
Then enter the following command (with /opt included)
du -hsc /home/e-smith /etc/e-smith/templates-custom \ /etc/e-smith/templates-user-custom /etc/ssh /root \ /etc/sudoers /etc/passwd /etc/shadow /etc/gshadow \ /etc/samba/secrets.tdb /etc/samba/smbpasswd /etc/smbpasswd /opt
or enter it as one long line without the \
du -hsc /home/e-smith /etc/e-smith/templates-custom /etc/e-smith/templates-user-custom /etc/ssh /root /etc/sudoers /etc/passwd /etc/shadow /etc/gshadow /etc/samba/secrets.tdb /etc/samba/smbpasswd /etc/smbpasswd /opt
Save the file and exit by pressing the following keys at the same time
Ctrl o Ctrl x
Make the file executable
chmod +x /opt/scripts/datasize
Run the command, which may take a few minutes to complete
/opt/scripts/datasize
You should see output similar to
151G /home/e-smith/ 76K /etc/e-smith/templates-custom/ 4.0K /etc/e-smith/templates-user-custom/ 152K /etc/ssh/ 289M /root/ 4.0K /etc/sudoers 8.0K /etc/passwd 4.0K /etc/shadow 4.0K /etc/gshadow 12K /etc/samba/secrets.tdb 4.0K /etc/samba/smbpasswd 98M /opt 151G total
Command line manual Restore from USB drive or memory stick
This is not the usual recommended way to do Restores, but if required it can be achieved.
From http://forums.contribs.org/index.php?topic=42012.msg196939#msg196939
Q) Is it possible to restore the server without a screen and CDROM ? Can this be done via ssh over the local network ?
A) A restore is possible using the following steps:
Fresh install of sme OS
Copy the previously made backup file, smeserver.tgz, to a fresh node eg
mount /media/usbdisk cp /media/usbdisk/Backup-date-folder/smeserver.tgz /
Login to command prompt as root
cd / signal-event pre-restore tar -C / -xzvf smeserver.tgz /sbin/e-smith/signal-event post-upgrade /sbin/e-smith/signal-event reboot
After reboot login to admin-console to change IPs to suit network
Save & Reboot
Delete backup file
cd / rm smeserver.tgz
System should now be online & operational
Different USB-Disks/Sticks and have been tested and it was observed that external USB-Drives with external Power run error free, but some USB-Sticks and USB-Disks powered by the USB port bus, can have issues (possibly low voltage/Hal-Daemon problems).
For Restore to a new other SME-Server with new Domain-FQDN (newserver.home.net).
delete old DNS entrys inside SME Database after a reconfigure+reboot!
#db host show (to list old entries!! like oldserver.home.net..) or #db host show > domainlist.txt (to export high count of entries and remove old with #db hosts delete oldserver.XXXX.XXX (XXX= domains.net mit prename oldserver!!) #signal-event post-upgrade #signal-event reboot
Check at Admin-Console(SSH-Login) -> Server Configuration -> IP if you have to change NEWSERVER-IP & FQDN!! Cause Restore set the IP & FQDN of OLDSERVER!! login again into https://yourserver/server-manager/ to check Domain settings
ready..
Restore on initial reboot after fresh OS install - How to Reset option
A full Restore from a backup file (eg located on a USB drive) is only recommended to be done immediately after installing a fresh sme server operating system. If this option is not selected on the first reboot, then the option is no longer available on subsequent reboots.
To make this option available, log in as root do the following:
config delete PasswordSet config setprop bootstrap-console Run Yes signal-event reboot
If using this technique, please ensure you have a "clean" system, with no users, groups or ibays etc configured.
If there are any users/groups/files added to the system since installation, then the results of the restore will be undefined and probably undesirable.
For example, if ibays were previously configured, then these will still be in the file system but they will not be displayed in server manager.
Backup and Restore concepts, issues and other information
The sme backup & restore concept is that only data (users & ibays) and configuration (including all mysql dbs and custom templates) is backed up & restored. All contribs (ie installed as rpms) are not backed up and normal templates (which are installed by the system or add on contribs) are not backed up.
A restore is done to a fresh installation of sme server, and then contribs are reinstalled. As configuration data is restored, you should not need to setup those reinstalled contribs again.
If you use any add on contrib for backup purposes eg Affa or Dar2 or whatever, then after doing a fresh reinstall of the sme OS, the first thing you do is reinstall the backup contrib and do some basic configuration manually. Then you would restore using that contrib, then you can reinstall all other contribs etc.
It's good admin practise to create an ibay specifically to house a copy of every rpm you install, so you can easily reinstall those from your backed up and restored copies, although with yum repos being available now, it's not so necessary to do this. It does provide a reminder of everything you have installed though, in case you forget. You should also do this if you wish to retain specific versions of rpms and source rpms, and also include copies of dependency rpms for good measure too.
The Affa contrib provides you with a report of missing rpms (if you enable that setting), which is the difference between the source server and the backup server, so you can easily see what needs reinstalling after a restore or Affa "rise", or install those on the Affa backup server in advance (if using a separate backup server).
Note that a tape backup backs up all file systems on the server, but only restores the "standard restore" files. See this list (above) http://wiki.contribs.org/Backup_server_config#Standard_backup_.26_restore_inclusions
A full restore from a tape backup to a freshly installed sme OS, will cause all sorts of issues. The restored server will start to boot up but then grind to a halt during the startup/post configuration processes.
If using alternative full backup and full restore methods, problems will be experienced when attempting to restore rpms which have executables/binaries installed in the normal Linux file system locations, that's why they should be reinstalled. If you restore any of /lib, /bin etc onto a running SME server, you are almost certain to have problems.
Also some contribs will need to be updated if you restore to a newer version of the sme OS, so if you restore old contribs, then you will break your server, in some cases.
If you wish to backup "everything" (including installed rpms), then the standard sme backup methods (backup to desktop and backup with dar), and most of the available backup contribs eg DAR2, Affa, etc, do not backup "everything". You need to perform a bare metal or disk clone backup to achieve that. A contrib that does that is Mondo, but it appears to have not been updated to be compatible with sme7.x. Another way of easily doing this is to swap out a RAID1 disk, and plug in a clean (empty) disk of exactly identical size or larger. There will only be a few minutes downtime while the disk is swapped, as hot swap is not currently supported. After the clean disk is inserted, then use the admin console to resync the RAID1 array. The system will happily run on one disk in degraded mode until the synchronisation is complete. Disk cloning software can also be used, but it must support RAID partitions and lvm volumes, and some cloning software does not. The Linux dd command will also copy every bit on a disk, to another mounted disk, but it will be slow and may take a long time (many many hours) for a disk with a lot of data. The server would ideally be disconnected from the local network and the Internet during this operation to prevent data changing on the server during the disk clone/backup process. Issue the command man dd for further information. Each of these methods has their own practical issues, choose the one that best suits your situation.
Add on scripts etc are best implemented via custom templates (which do get backed up), and any scripts can be put into /opt/xxx, and you then add /opt to your list of backup inclusions, or put them into an ibay if preferred, which does get backed up automatically.
The DAR2 contrib has a web interface that is configurable and allows you to create the disaster recovery backup (ie all data and configuration) required to rebuild a server, but you still need to reinstall add on contribs (rpms). It also allows selective backup jobs to be done, eg of certain ibays only, depending on your particular needs.
Cifs Error
When using the included backup functionality, or the DAR2 contrib, backing up to a windows remote share generates the following cifs errors. see bugzilla:7341
Feb 12 11:51:25 sme e-smith-bg: Connecting /mnt/dar2/backup to \\192.168.10.30\bkp Feb 12 11:51:25 sme kernel: CIFS VFS: Send error in SessSetup = -12 Feb 12 11:51:25 sme kernel: CIFS VFS: cifs_mount failed w/return code = -12 Feb 12 11:51:25 sme e-smith-bg: mount error(12): Cannot allocate memory Feb 12 11:51:25 sme e-smith-bg: Refer to the mount.cifs
The solution could be to modify the windows registry by the file below
Windows Registry Editor Version 5.00 [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management] "LargeSystemCache"=dword:00000001 [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\LanmanServer\Parameters] "Size"=dword:00000003
Save the content above in a file with the name W7-backup.reg and import it in the windows registry, restart the service or the Windows computer
Reference links
http://wiki.contribs.org/UpgradeDisk
http://forums.contribs.org/index.php?topic=30745.0
http://forums.contribs.org/index.php?topic=39969.0
