Difference between revisions of "Backup with dar"

From SME Server
Jump to navigationJump to search
(edited Overview)
(Minor edit for clarity)
 
(100 intermediate revisions by 21 users not shown)
Line 1: Line 1:
 
== Backup with dar using e-smith-backup ==
 
== Backup with dar using e-smith-backup ==
  
 +
http://wiki.contribs.org/SME_Server:Documentation:Administration_Manual:Chapter10#Backup_or_restore
  
−
==='''Overview'''===
+
[http://dar.linux.free.fr/ DAR homepage]
  
−
This contrib requires sme7.2. It is a replacement for the standard e-smith-backup rpm, and adds extra settings to the standard server manager backup or restore panel.
+
{{Note box|'''Important''' - Be aware of the concepts and issues as described in this section https://wiki.contribs.org/Backup_server_config#Backup_and_Restore_concepts.2C_issues_and_other_information}}
−
It can be configured to do daily full or incremental backup to a smbfs/cifs/nfs share on a LAN workstation or to a local USB disk.
 
−
An email notification will be sent to admin when the backup job finishes.
 
−
This contrib uses dar to perform backups, so you need to install the dar rpm as well.
 
  
−
==='''Installation'''===
+
===Typical scenarios===
  
−
Change (cd) to to a empty folder and do:  
+
Select ''Configure Workstation Backup'' and click perform
−
  wget http://mirror.contribs.org/smeserver/contribs/dmay/smeserver/7.x/dar-2.3.2-1.i386.rpm
+
 
−
  wget http://beta.free-eos.org/sme/RPMS/e-smith-backup-1.14.0-16eos1.el4.sme.noarch.rpm
+
====Windows Share====
−
  yum localinstall dar*.rpm
+
 
−
  yum localinstall e-smith-backup*.rpm
+
With Server backing up to a removable or shared drive in a workstation, with a weekly full backup on Friday and subsequent daily incremental backups & you wish to retain 2 backup sets on each disk, with backup to start at 8pm and finish at 8am.
−
+
 
 +
Backup is enabled.
 +
Backup is made on LAN workstation netshare via cifs
 +
Destination backup share folder is backup
 +
Login is backupuser
 +
Password is ********
 +
Number of rotating backup sets is 2
 +
Number of daily backups contained in each set is 7
 +
Compression level (0-9) of backup is 4
 +
Daily backup occurs at 20:00
 +
Each daily backup session is cleanly timed out after 12h except full backups which are cleanly timed out after 24 hours
 +
Full backup session (new backup sets) is allowed only on Friday
 +
 
 +
====USB Disk attached to Server====
 +
For help with USB Disk preparation see [[:USBDisks]]
 +
 
 +
Backup is Enabled.
 +
Backup is made on local USB disk
 +
Destination backup share folder is media/Diskname
 +
Number of rotating backup sets is 2
 +
Number of daily backups contained in each set is 14
 +
Compression level (0-9) of backup is 4
 +
Daily backup occurs at 18:30
 +
Each daily backup session is cleanly timed out after 8 hours.
 +
Full backup sessions (new backup set) are allowed everyday
 +
 
 +
 
 +
====More common scenarios & explanation of backup selection settings====
 +
 
 +
Here are some typical scenarios, keep in mind the choice of full backup on a specified day or every day will also interact with these settings.
 +
 
 +
 
 +
If you wanted to do a full backup every day, & only retain 1 historical backup, then select
 +
 
 +
Number of daily backups contained in each set is 1.
 +
 
 +
Number of rotating backup sets is 1.
 +
 
 +
 
 +
If you wanted to do a full backup once every 7 days, then daily incremental backups every other day, & only retain 1 historical backup, select
 +
 
 +
Number of daily backups contained in each set is 7.
 +
 
 +
Number of rotating backup sets is 1.
 +
 
 +
 
 +
If you wanted to do a full backup once every 14 days, then daily incremental backups every other day, & only retain 1 historical backup, select
 +
 
 +
Number of daily backups contained in each set is 14.
 +
 
 +
Number of rotating backup sets is 1.
 +
 
 +
 
 +
If you wanted to do a full backup once every 28 days, then daily incremental backups every other day, & only retain 1 historical backup, select
 +
 
 +
Number of daily backups contained in each set is 28.
 +
 
 +
Number of rotating backup sets is 1.
 +
 
 +
 
 +
For any of the above, if you want to retain more historical backup sets, then change the
 +
 
 +
Number of rotating backup sets to 2 or 3 or 4 as desired
 +
 
 +
 
 +
So let's say the full backup is set to run on Friday only, & the backup time is set for 11pm.
 +
 
 +
On the first Friday night at 11pm a full backup is done, this may take many hours or even time out if there is a lot of data to backup.
 +
 
 +
On the first Saturday night at 11pm if the full backup did not complete the night before (timed out as per time out setting or reached 24 hours run time), then the full backup is continued on from where it left off.
 +
 
 +
If the full backup did complete, then an incremental backup is done, so only the additions & changes of data are added to the first full backup.
 +
 
 +
These are added as incremental backups, in a seperate incremental backup file with a date identifier.
 +
 
 +
Then every day of the backup period (being the Number of daily backups contained in each set), ie 7, 14 or 28 days, an incremental backup is done.
 +
 
 +
Then on the Friday (being either day 8, 15 or 29 in examples above) at 11pm a new full backup is done.
 +
 
 +
After that incremental backups are done each day, & so the cycle continues on.
 +
 
 +
 
 +
Note, when a full backup is done:
 +
 
 +
If Number of rotating backup sets is set to 1, then the original (first) backup is deleted.
 +
 
 +
If Number of rotating backup sets is set to 2, then the original (first) backup set is retained & a new backup set is created/started.
 +
 
 +
If Number of rotating backup sets is set to 3, then the first & second backup sets are retained & a new backup set is created/started.
 +
 
 +
If Number of rotating backup sets is set to 4, then the first, second & third backup sets are retained & a new backup set is created/started.
 +
 
 +
.... & so on.
 +
 
 +
Note that if a full restore is needed, the full & incremental backups are used, so all data is restored.
 +
 
 +
 
 +
A "practical" backup approach may be to configure for a Full backup every 7 days & daily incremental backups in between, with 2 rotating sets, so a new full backup is created each Friday & you retain a complete 7 days worth of backups for historical purposes.
 +
 
 +
You could even extend that to retaining 4 sets of historical backups, in case you wanted to restore some individual files from 25 days ago.
 +
 
 +
Another approach is to have 2 backup USB disks, & swap the disk every Friday (during the day) so the new full backup (on Friday night) is done to the alternate USB drive, and that alternate drive is used for the rest of the 7 days period for daily incremental backups.
 +
 
 +
How often you want to create a full backup & how many daily incrementals you want to have in a set & how many historical sets & whether you swap disks off site etc, is really a part of your overall backup protocol or strategy for the site/server in question.
 +
 
 +
===Backup Sets===
 +
The number of increments in a set is only equal to the "Daily backups in each set"
 +
if a multiple of 7. For instance choose 27, and you only get 21 increments,
 +
whereas 28 gives 28 increments. Or choose 3, and it gives 7 increments.
 +
(This assumes that "Full backup is allowed on" is not set to everyday).
 +
 
 +
If you want the rotation, and full backup, to occur on a specified day, then
 +
choose that day in the panel, and choose a multiple of 7 for "Daily backups in
 +
each set".
 +
 
 +
If you want a fixed number of increments, less than a multiple of 7, in each set, then choose "Full backup
 +
is allowed on" = Everyday. eg Daily backups in each set = 6 will result in 1 Full backup and 5 incremental backups, on each full backup all backups are deleted including the incrementals.
 +
 
 +
 
 +
The above descriptions can relate to unusual combination of settings, where the number of incrementals does not match or equate to the full backup period.
 +
It is probably simpler to use (& to understand) setting values that are relative to each other eg weekly (7 day) full backup with 7 incrementals.
 +
 
 +
===Backup timeouts explained===
 +
Even if you choose not to timeout your full backups, a full backup cannot
 +
exceed 24h (or the cron will launch a conflicting backup job).
 +
 
 +
So when you choose "Don't timeout full backup sessions", there is a default
 +
timeout just before 24h. Then the full backup can stop cleanly, and the new
 +
(incremental) backup will continue the backup.
 +
 
 +
The real goal of setting a timeout+"Don't timeout full backup sessions" is for
 +
big user files systems, when we want that the backup session occurs only the
 +
night on the week days (we need the cpu and the bandwith for something else
 +
than backuping, and if we backup on the time there is too much activity, the
 +
risk of a failing backup with dar grows up, because a file can be modified
 +
during his backup), and all the days on saturday and sunday (for most common
 +
case).
 +
 
 +
We can set :
 +
 
 +
Backup time : 10:00 PM
 +
DaysInSet : 7
 +
FullDay on friday or saturday
 +
IncOnlyTimeOut : yes
 +
Timeout : 8 (backup stops at 6:00 AM on week days)
 +
 
 +
On monday morning we can have fully backuped our server, even if the time to do
 +
it is about 32 hours (24+8). On the week days the server and the lan are are
 +
not busy with backups between 6:00 AM and 10:00 PM
 +
 
 +
For servers that have large amounts of data, 200 GB+, you may see the following error with incremental backups
 +
  Aborting program. User refused to continue while asking:
 +
/mnt/smb/server.2activepr.ro/set0/full-20090404.1.dar is required for further
 +
operation, please provide the file.
 +
 
 +
To workaround set timeout to 23 hrs, this allows the backup to timeout cleanly, it will restart and attempt to complete the first full backup on the next run.
 +
 
 +
 
 +
===Using the same backup share to backup several SME servers===
 +
 
 +
If you have more than one SME server on your LAN, you can use the same share (nfs or cifs) to backup all your servers. Each server will backup his data in a specific directory (the name of this directory is the FDQN of the server).
 +
 
 +
 
 +
===Backup disk size limits workaround===
 +
 
 +
{{Warning box|The method described here permits to get daily backup work on a support with not enough space to do a new daily full backup without, before, deleting the current backup. So it must be considered with caution : if the backup fails, you'll be whithout any backup available. As explained, use this method only if security of your data is not essential, or buy a larger disk for storage.}}
 +
 
 +
If having issues with the size of removable backup disks (eg 120Gb IDE on a workstation) versus the resultant backup data size (eg 120Gb of data compressing to approx 50Gb), then you can manually setup a cron job to delete the old backup file(s) before saving the new backup file(s).
 +
 
 +
The specific issue experienced was being able to get one full backup onto a disk, and a weeks worth of daily incrementals, but the next weekly full backup failed, due to the need to save the new backup in the tmp folder on the removable disk, before deleting the old backup files. There was not enough space on the disk (size) being used.
 +
 
 +
To workaround this, create a cron job and put it in /etc/cron.d with a script in /opt/scripts/XXX, which mounts the drive and deletes the old backup just before the new full backup commences.
 +
This will delete the old backup before the new backup commences, which is different to how the backup is designed to work by default. It is necessary to forgot the security of retaining the old backup until the new one has completed, in order to use the available hardware (smaller disk).
 +
 
 +
 
 +
Create the deletion script
 +
nano -w /opt/scripts/deletebkp1
 +
 
 +
Enter the following (using your share name, user, pasword & domain)
 +
umount /mnt/smb
 +
/bin/mount -t cifs //stationXX/bkp1 /mnt/smb -o username=XXXXXX,password=XXXXXXXXXXXXX
 +
rm -f -R /mnt/smb/servername.yourdomain.com/
 +
rm -f -R /mnt/smb/tmp_dir/
 +
 
 +
Then save & exit
 +
Ctrl o
 +
Ctrl x
 +
 
 +
 
 +
Create the cron job
 +
nano -w /etc/cron.d/deletebkp1
 +
 
 +
Enter the following (setting the time & day to be just before your scheduled backup time), see below.
 +
The following values suit a scheduled full backup for 2.00am on Saturday morning (ie Friday evening), where the old backup gets deleted at 1:50am
 +
 
 +
  50    1  *    *   6    root  . /opt/scripts/deletebkp1
 +
 
 +
Then save & exit
 +
Ctrl o
 +
Ctrl x
 +
 
 +
 
 +
Cron entries format is as follows:
 +
  +-----------------------Minute            (0-59)
 +
|    +-------------------Hour of Day      (0-23)
 +
|    |    +---------------Day of Month    (1-31)
 +
|    |    |    +-----------Month of Year  (1-12)
 +
|    |    |    |    +-------Day of Week  (0=Sun,6=Sat)
 +
 
 +
50  1    *    *   6    root  . /opt/scripts/deletebkp1
 +
 
 +
===Howto set the backup file slice size===
  
−
To prevent updates from overwriting the modified rpm do:
+
The default slice or part setting is 700Mb (to suit CD's).
−
echo e-smith-backup > /etc/e-smith/db/yum_repositories/defaults/smeupdates/Exclude
+
The slice size can be changed for example to suit 4.7Gb DVD's, by making 3 parts fit on to a DVD, so the required slice size is 1529Mb (allowing for overheads & real data size).
−
/etc/e-smith/events/actions/initialize-default-databases
 
−
expand-template /etc/yum.conf
 
−
Future updates to this rpm will need to be done manually.
 
  
 +
Set the Slice size in the config DB
 +
config setprop backupwk Slice 1529M
  
−
Open server manager & select the Backup or restore panel
+
Then expand the template with
−
Click on the the pull down menu for Select an action and you will see extra choices which have been added ie
+
expand-template /etc/dar/DailyBackup.dcf
  
−
Configure workstation backup
+
The next time the backup runs, the resultant backup file will be split into 1.529Gb part sizes.
  
−
Verify workstation backup
+
===Adding/Excluding Directories and Files from the backup list===
  
−
Restore from workstation
+
{{Warning box|The e-smith-backup panel is designed to be used as a full disaster recovery backup & restore method for your sme server, and as such relies on the correct directories being included in the backup. Do not delete any of the default directories from 40go-into as these are all essential for a successful full Restore of your backup to a clean install of the sme server operating system (you will need to reinstall add on apps or contribs). You can safely add directories eg /opt (without leading '/' ), as indicated, but that should typically be the extent of changes made.}}
 +
====Adding Files and Directories====
 +
The default backup does not include /opt
  
−
Selective file restore from workstation
+
To add /opt to the backup inclusion list, create a custom template fragment and add opt to the list.
  
−
Select "Configure workstation backup" and configure it. Further information is in the Detailed Information section below.
+
Create custom template fragment to add needed files and directories
 +
mkdir -p /etc/e-smith/templates-custom/etc/dar/DailyBackup.dcf
 +
 +
nano -w /etc/e-smith/templates-custom/etc/dar/DailyBackup.dcf/41go-into
 +
 +
--go-into opt
  
−
==='''Manual upgrade information'''===
+
Then expand the template
 +
expand-template /etc/dar/DailyBackup.dcf
  
−
Note that the commands issued earlier prevent the modified e-smith-backup (with dar) rpm from being replaced by normal upgrades done using either the software installer panel or yum update.
 
  
−
When changes to the base sme server e-smith-backup rpm occur, an updated modified version of the e-smith-backup with dar rpm will be released and users will need to manually update it. Download the new modified version of e-smith-backup from beta.free-eos.org & follow the install instructions above.
+
Check your backup (eg using the [[Midnight Commander plugin for DAR|mc dar plugin]]) to see that /opt is now included.
 +
It is also prudent to do a test full restore and verify that /opt (& all other backed up directories & files) are restored, as anticipated.
 +
If something is wrong with your backup or restore procedures, then required files may not be backed up and restored, so it is wise to do a test full backup & restore to fully prove your procedures are working correctly.
  
 +
{{Note box|<tt>--include <mask></tt> will include single files into the backup when using a path (without leading '/' ) and file name for the mask, or all files matching the mask when using wildcards. Like:
 +
  --include path/to/file/sampleFileName.txt
 +
  --include "*.backMeUp"
 +
<tt>--go-into <path></tt> will include entire directories with all of it's contents (files and subdirs) into the backup. Like:
 +
  --go-into path/to/dir/sampleDirName
 +
}}
  
−
==='''Detailed Information'''===
+
====Excluding Files and Directories====
 +
As an example let's say you have an ibay called 'mirror' you don't want backed-up. To exclude this ibay from the backup list, create a custom template fragment and exclude it from the the list.
  
 +
Create custom template fragment to exclude the unneeded files and directories
 +
  mkdir -p /etc/e-smith/templates-custom/etc/dar/DailyBackup.dcf
 +
  nano -w /etc/e-smith/templates-custom/etc/dar/DailyBackup.dcf/45prune
 +
 
 +
  --prune home/e-smith/files/ibays/mirror/cgi-bin
 +
  --prune home/e-smith/files/ibays/mirror/files
 +
  --prune home/e-smith/files/ibays/mirror/html
  
−
====''What is e-smith-backup with dar ?''====
+
Then expand the template
 +
  expand-template /etc/dar/DailyBackup.dcf
  
−
e-smith-backup with dar is an upgrade of current e-smith-backup sme core rpm which provides panels for daily full or incremental backup on LAN workstation (via nfs, smbfs or cifs) or local usb disk, and full or selective restore  with use of dar program.
+
Again, check your backup (eg using the mc dar plugin) to see that the ibay is not backed-up any more.
  
 +
{{Note box|<tt>--exclude <mask></tt> will exclude single files when using a path (without leading '/' ) and file name for the mask or all files matching the mask when using wildcards, from the backup. Like:
 +
  --exclude path/to/file/sampleFileName.txt
 +
  --exclude "*.NoBackup"
 +
<tt>--prune <path></tt> will exclude entire directories with all of it's contents (files and subdirs) from the backup. Like:
 +
  --prune path/to/dir/sampleDirName
 +
}}
  
−
====''What provides e-smith-backup with dar ?''====
+
===Use WOL to power on Workstation===  
 +
{{Note box|This is a new feature and not implimented in SME 8.0. It will be available from SME 8.1}}
 +
Wake On Lan can be used to power up the backup target workstation before starting the backup.
  
−
e-smith-backup with dar completes core e-smith-backup with four panels :
+
To use WOL there simply needs to be a new variable added to the backupwk section of the configuration database, nothing else is required. This variable holds the MAC address of the target workstation, if the MAC address is 00:4E:89:F5:FD:2B use:
  
−
- configure daily full or incremental backup to smbfs/cifs/nfs share on a LAN workstation or on local usb disk
+
db configuration setprop backupwk SmbHostMAC 00:4E:89:F5:FD:2B
−
- verify backup on LAN workstation or local usb disk
 
−
- restore full backup from LAN workstation or local usb disk
 
−
- selective restore of files or directories from LAN workstation or local usb disk
 
  
−
New backup options are provided with additional perl functions included in backup panel script, a new perl backup script /etc/e-smith/events/actions/workstation-backup-dar, and a new backupwk record in configuration db. All are written with the same logic as present backup scripts.
+
There is also the ability to specify how long to wait between the WOL packet being sent and attempting to start the backup process. The default wait time is 300 seconds, this can be varied via another configuration database setting. It is recommended to not set the wait period below 300 seconds. To set the wait time to 600 seconds:
  
 +
db configuration setprop backupwk SmbHostDelay 600
  
−
====''What are specific problems of SME Server LAN backups managed by e-smith with dar ?''====
+
The target system must support, and be set up to respond to, the WOL "magic packets". The network infrastructure must also support WOL packets. WOL should work across the local network without problem. WOL packets are not routeable so it won't work across the internet without additional support from hardware and/or software such as a VPN tunnel. Getting WOL to work across the internet is beyond the scope of this documentation but there are plenty of resources available elsewhere. Most wireless connected devices do not work with WOL.
  
−
With experience of more than four years backuping sme servers with dar on LAN workstations (in production environments) the most important of encountered problems is that, when the backup session is started, you cannot rely upon a perfect working of the LAN or the distant workstation.
+
Note that there is no checking that the supplied MAC address is the correct one for the IP address/hostname of the target workstation. MAC address's are effectively static but IP address's/hostnames can change particularly if the target gets its IP address via DHCP. It is quite possible to have the correct target woken up and be ready but the backup to fail because the IP address has changed. However SME's DHCP server very rarely changes the IP address of a given MAC address.
−
Inevitably, one day the LAN is too busy and/or the connection to the distant share hacks. After the failure sme may keep zombies backup or mount processes, and with some contribs you can be left with incomplete or buggy backup until hopped next good daily backup. These problems are problems of real world with LAN, that backup scripts must take into account.
 
  
−
Another common problem with backup through a LAN is that your lan disponibility for backup flow is not 24/24, but more probably something like "only from 20h to 6h, except on sunday where nobody works on the LAN..." So in case of large volume of data to backup, beeing able to backup all the datas during the week-end and doing only incremental backup on the week days really makes sense.
+
To stop using WOL simply delete the SmbHostMAC variable:
  
−
All these problems are considered with e-smith-backup with dar.
+
  db configuration delprop backupwk SmbHostMAC
−
Scripts verify the absence of old share mounts, and alert admin about such situation. Test is made that the distant backup share is available and writable. Of course with daily backup a maximum backup session time of only 24 h is available, but if this limit occurs e-smith-backup with dar cleanly stops the backup session  (maybe all is ok and the backup has not enough time to complete), the lan share is unmounted, and daily process can restart anyway. If you have enabled incremental backup and the last backup (full or incremental) was not completed because of a timeout, the backup can then safely restart and complete in an incremental way.
 
  
−
The admin can also set a timeout limit which keeps the session inside a specified number of hours, excluding traffic of LAN backup sessions when users work on the LAN.
+
and to be tidy if the optional delay parameter has been set:
  
−
Admin user can also set backup configuration so that this time limit occurs only on incremental backup sets. Then if you have a lot of data and your full backup time exceeds time of LAN inactivity, you can always backup all your data each end of week and preserve LAN bandwith for users. If you  program a full backup on saturday night, you get 24 hours to do full backup and even if the full backup time exceeds 24 hours, the first incremental backup on sunday night will backup remaining data. You begin your week with full data saved, and incremental backup session time is limited to 6 or 8 hours during week days.
+
  db configuration delprop backupwk SmbHostDelay
  
−
Full backup sessions can also be allowed on every day, or limited to a specific day of week.
+
===How to start a one-off backup===
  
 +
To backup up to workstation as a one-off, not a scheduled event, run
  
−
====''Are other new functions provided ?''====
+
/sbin/e-smith/do_backupwk
  
−
Yes. Aside use of session timeout, the first is incremental backup.
+
However note that scheduled backups must be enabled for the above command to work.
−
This means that you can backup and restore data for the period of time you want : one day, three days, one week, one month, 100 days... and restore your system at any state it was during this period of time. This probably has no utility to do full restore of the system as it was one month ago, but restoring a file lost by a user two or three weeks ago can be useful. And restoring a safe system more than one day old can be needed.
 
  
−
The second function is keeping more than one set of backup (a set is full backup data and all datas of next daily incremental backups) with automatic rotation. e.g. you can do only nightly full backups but keep three sets of backup for security reason  (as being able to restore the system as it was 72h ago).
+
== Some notes ==
 +
Some things to keep in mind when using SME-Server backup:
  
−
The third function is selective restore of any backuped file or directory, exactly as it was for any of the backuped days in your sets. Not only you can restore a lost file at his last state, but also say : make restore of the most recent version of the file before this given date... Selective restore is not an easy thing to manage by hand, and providing this in a simple way with panels is useful. Dar permits to manage selective restore and e-smith-backup with dar panels tries to keep this function as simple as possible to use.  
+
=== Backup file slices are NOT autonomous ===
 +
The catalogue of the contents of the backup is put in the last slice. If for some reason the last slice can not be written (e.g. backup medium is full or network failure), is lost or corrupt, the slices already saved are useless.
  
 +
=== Naming of the backups on the workstation share or USB disk ===
  
−
====''I read about problems about smbfs or cifs support with sme, and nfs packages are not on my sme...''====
+
* For a SME server all backups sets are put in a directory which has the FQDN of the server as a name, ie '''hostname.domain/'''
  
−
Before sme 7.2, smb and kernel versions of sme have bugs in smb mount and cifs support.
+
* All backups of a set are put in a subdirectory with the name of the set, ie. '''hostname.domain/set''0''/''', '''hostname.domain/set''1''/''', etc.
−
For smb mount bug, e-smith-backup with dar includes necessary work-around.
 
−
For cifs support, corrected 2.6.9-55 kernel is packaged with sme 7.2.
 
−
For nfs support, e-smith-backup doesn't need install of any nfs package on the sme server (not even portmap).
 
  
 +
* During a backup session, all session files are temporary put in the directory '''hostname.domain/tmp_dir/'''
  
−
==='''Practical setup tips'''===
+
* In a set the full backup is named '''full-yyyymmdd'''
  
−
Change these to suit your particular situation.
+
* In a set the incremental backups are named '''inc''1''-yyyymmdd''', '''inc''2''-yyyymmdd''', etc.  
  
−
In a typical scenario with Windows workstations backing up to a removable or attached drive in a workstation, with a weekly full backup on Friday and subsequent daily incremental backups & you wish to retain 2 backup sets on each disk, with backup to start at 8pm and finish at 8am, use the following settings.
+
* In a full backup the files (slices) are named '''full-yyyymmdd.1.dar''', '''full-yyyymmdd.2.dar''', etc.
  
−
Backup is enabled.
+
* In an incremental backup the files (slices) are named '''inc''n''-yyyymmdd.1.dar''', '''inc''n''-yyyymmdd.2.dar''', etc.
  
−
Backup is made on LAN workstation netshare via smbfs
+
All backup files have allways distinct names. If the backup files are moved elsewhere for archiving, you must recreate the directory structure before doing a restore with the server manager.
  
−
Destination backup share folder is backup
+
=== cifs - mount error 20 = Not a directory ===
  
−
Login is backupuser
+
Some network drives don't function correctly with Backup with DAR and the cifs file system. The mount command returns "mount error 20 = Not a directory". See [http://bugs.contribs.org/show_bug.cgi?id=4923 Bug 4923].
  
−
Password is ********
+
This can be worked around by adding the "nounix" option to the mount command in the /etc/e-smith/events/actions/workstation-backup-dar file.
  
−
Number of rotating backup sets is 2
+
{{Warning box|This change may not be preserved across updates/upgrades to the Backup with Dar sub-system or to SME Server itself.
  
−
Number of daily backups contained in each set is 7
+
An update/upgrade may stop your backups working and require this change to be made again.}}
  
−
Compression level (0-7) of backup is 4
+
Edit the above file and find the line:
  
−
Daily backup occurs at 20:00
+
$err = qx(/bin/mount -t cifs "$smbhost:$smbshare" $mntdir -o user=$login,pass=$password 2>&1);
  
−
Each daily backup session is cleanly timed out after 12h except full backups which are cleanly timed out after 24 hours
+
and change it to:
  
−
Full backup session (new backup sets) is allowed only on Friday
+
$err = qx(/bin/mount -t cifs "$smbhost:$smbshare" $mntdir -o nounix,user=$login,pass=$password 2>&1);
  
 +
Save the file.
  
−
==='''Removal/uninstall information'''===
+
== Manually restoring from dar archive created by e-smith-backup ==
  
−
The base sme server code is in the modified e-smith-backup rpm, so no changes are made to the standard core functionality of the sme server backup routines. The modified rpm can be left installed, just do not use any of the additional functionality.
+
Restores can be done for:
  
−
To fully remove the rpm & configuration from your system and reinstall the base rpm do the following steps. Check the current version of e-smith-backup (in the smeos repository location listed below) and replace the code appropriately.
+
-the entire system.
  
−
rpm -e --nodeps e-smith-backup
+
-a selected file only.
−
mkdir -p /newfolder
 
−
wget http://distro.ibiblio.org/pub/linux/distributions/smeserver/releases/7/smeos/i386/SME/RPMS/e-smith-backup-1.14.0-16.el4.sme.noarch.rpm
 
−
yum localinstall e-smith-backup*.rpm
 
−
signal-event post-upgrade
 
−
reboot
 
  
−
Check for any orphaned cron jobs & delete them if necessary
+
-a selected directory and all of its contents.
−
rm /etc/cron.d.backupwk
 
  
−
Edit the smeupdates repository default Exclude file & remove the entry for e-smith-backup (that was made during install)
+
The entire system and a selected file only can be done through the server-manager.
−
pico /etc/e-smith/db/yum_repositories/defaults/smeupdates/Exclude
 
  
−
Then remove e-smith-backup & save the changes (Ctrl + o then Ctrl + x), followed by
+
The selected directory can be done via shell.
 +
(You will have to adjust for your particular situation.)
  
−
  /etc/e-smith/events/actions/initialize-default-databases
+
First, MOUNT THE BACKUP DRIVE (if required):
−
expand-template /etc/yum.conf
+
  mount /dev/sdc1 /media/BackupDrive1/
  
−
The standard Software installer panel/yum update process will now automatically update e-smith-backup if a new version is released.
+
You can mount the configured CIFS/SMB destination using<syntaxhighlight lang="bash" >
 +
mount -t cifs //$(config getprop backupwk SmbHost)/$(config getprop backupwk SmbShare) $(config getprop backupwk Mount) -o credentials=/etc/dar/CIFScredentials
 +
</syntaxhighlight>List available backups<syntaxhighlight  lang="bash" >
 +
find $(config getprop backupwk Mount)/$(config get SystemName).$(config get DomainName) -name "*.1.dar" | sed s/\.1\.dar$//
 +
</syntaxhighlight>
  
 +
Then, RESTORE THE DIRECTORY:
 +
dar -x /media/BackupDrive1/server.domain.local/set2/full-201408092200 -N -R ./ -w -g home/e-smith/files/ibays/share_data/files
  
−
Check the versions of rpms installed (& if installed) with
+
This will restore the directory /home/e-smith/files/ibays/share_data/files to you current directory. If your current directory is /tmp/, the files will be recovered to /tmp/home/e-smith/files/ibays/share_data/files. This will make you able to search through the recovered files and, if needed, move them to their proper location.
−
rpm -qa dar
 
−
rpm -qa e-smith-backup
 
  
 +
{{Warning box|Be sure to use -R ./ (the dot) to restore in the current dir, otherwise you'll restore files/dirs in their original path}}
  
−
Reference: http://forums.contribs.org/index.php?topic=37922.0
+
You may also have to RESTORE ANY INCREMENTS:
 +
dar -x /media/BackupDrive1/server.domain.local/set2/inc-001-201408102200.1.dar -N -R ./ -w -g home/e-smith/files/ibays/share_data/files
 +
dar -x /media/BackupDrive1/server.domain.local/set2/inc-002-201408112200.1.dar -N -R ./ -w -g home/e-smith/files/ibays/share_data/files
 +
dar -x /media/BackupDrive1/server.domain.local/set2/inc-003-201408122200.1.dar -N -R ./ -w -g home/e-smith/files/ibays/share_data/files
 +
dar -x /media/BackupDrive1/server.domain.local/set2/inc-004-201408132200.1.dar -N -R ./ -w -g home/e-smith/files/ibays/share_data/files
 +
 
 +
{{Note box|Unmount the backup destination after any manual exploration:
 +
  umount $(config getprop backupwk Mount)}}
  
 
----
 
----
−
[[Category:Contrib]]
+
 
 
[[Category: Backup]]
 
[[Category: Backup]]
 +
[[Category: Howto]]

Latest revision as of 19:42, 12 February 2022

Backup with dar using e-smith-backup

http://wiki.contribs.org/SME_Server:Documentation:Administration_Manual:Chapter10#Backup_or_restore

DAR homepage


Important.png Note:
Important - Be aware of the concepts and issues as described in this section https://wiki.contribs.org/Backup_server_config#Backup_and_Restore_concepts.2C_issues_and_other_information


Typical scenarios

Select Configure Workstation Backup and click perform

Windows Share

With Server backing up to a removable or shared drive in a workstation, with a weekly full backup on Friday and subsequent daily incremental backups & you wish to retain 2 backup sets on each disk, with backup to start at 8pm and finish at 8am. 

Backup is enabled.
Backup is made on LAN workstation netshare via cifs
Destination backup share folder is backup
Login is backupuser
Password is ********
Number of rotating backup sets is 2
Number of daily backups contained in each set is 7
Compression level (0-9) of backup is 4
Daily backup occurs at 20:00
Each daily backup session is cleanly timed out after 12h except full backups which are cleanly timed out after 24 hours
Full backup session (new backup sets) is allowed only on Friday

USB Disk attached to Server

For help with USB Disk preparation see USBDisks 

Backup is Enabled.
Backup is made on local USB disk
Destination backup share folder is media/Diskname
Number of rotating backup sets is 2
Number of daily backups contained in each set is 14
Compression level (0-9) of backup is 4
Daily backup occurs at 18:30
Each daily backup session is cleanly timed out after 8 hours.
Full backup sessions (new backup set) are allowed everyday


More common scenarios & explanation of backup selection settings

Here are some typical scenarios, keep in mind the choice of full backup on a specified day or every day will also interact with these settings.


If you wanted to do a full backup every day, & only retain 1 historical backup, then select

Number of daily backups contained in each set is 1.

Number of rotating backup sets is 1.


If you wanted to do a full backup once every 7 days, then daily incremental backups every other day, & only retain 1 historical backup, select

Number of daily backups contained in each set is 7.

Number of rotating backup sets is 1.


If you wanted to do a full backup once every 14 days, then daily incremental backups every other day, & only retain 1 historical backup, select

Number of daily backups contained in each set is 14.

Number of rotating backup sets is 1.


If you wanted to do a full backup once every 28 days, then daily incremental backups every other day, & only retain 1 historical backup, select

Number of daily backups contained in each set is 28.

Number of rotating backup sets is 1.


For any of the above, if you want to retain more historical backup sets, then change the

Number of rotating backup sets to 2 or 3 or 4 as desired


So let's say the full backup is set to run on Friday only, & the backup time is set for 11pm.

On the first Friday night at 11pm a full backup is done, this may take many hours or even time out if there is a lot of data to backup.

On the first Saturday night at 11pm if the full backup did not complete the night before (timed out as per time out setting or reached 24 hours run time), then the full backup is continued on from where it left off.

If the full backup did complete, then an incremental backup is done, so only the additions & changes of data are added to the first full backup.

These are added as incremental backups, in a seperate incremental backup file with a date identifier.

Then every day of the backup period (being the Number of daily backups contained in each set), ie 7, 14 or 28 days, an incremental backup is done.

Then on the Friday (being either day 8, 15 or 29 in examples above) at 11pm a new full backup is done.

After that incremental backups are done each day, & so the cycle continues on.


Note, when a full backup is done:

If Number of rotating backup sets is set to 1, then the original (first) backup is deleted.

If Number of rotating backup sets is set to 2, then the original (first) backup set is retained & a new backup set is created/started.

If Number of rotating backup sets is set to 3, then the first & second backup sets are retained & a new backup set is created/started.

If Number of rotating backup sets is set to 4, then the first, second & third backup sets are retained & a new backup set is created/started.

.... & so on.

Note that if a full restore is needed, the full & incremental backups are used, so all data is restored.


A "practical" backup approach may be to configure for a Full backup every 7 days & daily incremental backups in between, with 2 rotating sets, so a new full backup is created each Friday & you retain a complete 7 days worth of backups for historical purposes.

You could even extend that to retaining 4 sets of historical backups, in case you wanted to restore some individual files from 25 days ago.

Another approach is to have 2 backup USB disks, & swap the disk every Friday (during the day) so the new full backup (on Friday night) is done to the alternate USB drive, and that alternate drive is used for the rest of the 7 days period for daily incremental backups.

How often you want to create a full backup & how many daily incrementals you want to have in a set & how many historical sets & whether you swap disks off site etc, is really a part of your overall backup protocol or strategy for the site/server in question.

Backup Sets

The number of increments in a set is only equal to the "Daily backups in each set" if a multiple of 7. For instance choose 27, and you only get 21 increments, whereas 28 gives 28 increments. Or choose 3, and it gives 7 increments. (This assumes that "Full backup is allowed on" is not set to everyday).

If you want the rotation, and full backup, to occur on a specified day, then choose that day in the panel, and choose a multiple of 7 for "Daily backups in each set".

If you want a fixed number of increments, less than a multiple of 7, in each set, then choose "Full backup is allowed on" = Everyday. eg Daily backups in each set = 6 will result in 1 Full backup and 5 incremental backups, on each full backup all backups are deleted including the incrementals.


The above descriptions can relate to unusual combination of settings, where the number of incrementals does not match or equate to the full backup period. It is probably simpler to use (& to understand) setting values that are relative to each other eg weekly (7 day) full backup with 7 incrementals.

Backup timeouts explained

Even if you choose not to timeout your full backups, a full backup cannot exceed 24h (or the cron will launch a conflicting backup job).

So when you choose "Don't timeout full backup sessions", there is a default timeout just before 24h. Then the full backup can stop cleanly, and the new (incremental) backup will continue the backup.

The real goal of setting a timeout+"Don't timeout full backup sessions" is for big user files systems, when we want that the backup session occurs only the night on the week days (we need the cpu and the bandwith for something else than backuping, and if we backup on the time there is too much activity, the risk of a failing backup with dar grows up, because a file can be modified during his backup), and all the days on saturday and sunday (for most common case).

We can set : 

Backup time : 10:00 PM
DaysInSet : 7
FullDay on friday or saturday
IncOnlyTimeOut : yes
Timeout : 8 (backup stops at 6:00 AM on week days)

On monday morning we can have fully backuped our server, even if the time to do it is about 32 hours (24+8). On the week days the server and the lan are are not busy with backups between 6:00 AM and 10:00 PM

For servers that have large amounts of data, 200 GB+, you may see the following error with incremental backups 

Aborting program. User refused to continue while asking:
/mnt/smb/server.2activepr.ro/set0/full-20090404.1.dar is required for further
operation, please provide the file.

To workaround set timeout to 23 hrs, this allows the backup to timeout cleanly, it will restart and attempt to complete the first full backup on the next run.


Using the same backup share to backup several SME servers

If you have more than one SME server on your LAN, you can use the same share (nfs or cifs) to backup all your servers. Each server will backup his data in a specific directory (the name of this directory is the FDQN of the server).


Backup disk size limits workaround

Warning.png Warning:
The method described here permits to get daily backup work on a support with not enough space to do a new daily full backup without, before, deleting the current backup. So it must be considered with caution : if the backup fails, you'll be whithout any backup available. As explained, use this method only if security of your data is not essential, or buy a larger disk for storage.


If having issues with the size of removable backup disks (eg 120Gb IDE on a workstation) versus the resultant backup data size (eg 120Gb of data compressing to approx 50Gb), then you can manually setup a cron job to delete the old backup file(s) before saving the new backup file(s).

The specific issue experienced was being able to get one full backup onto a disk, and a weeks worth of daily incrementals, but the next weekly full backup failed, due to the need to save the new backup in the tmp folder on the removable disk, before deleting the old backup files. There was not enough space on the disk (size) being used.

To workaround this, create a cron job and put it in /etc/cron.d with a script in /opt/scripts/XXX, which mounts the drive and deletes the old backup just before the new full backup commences. This will delete the old backup before the new backup commences, which is different to how the backup is designed to work by default. It is necessary to forgot the security of retaining the old backup until the new one has completed, in order to use the available hardware (smaller disk).


Create the deletion script 

nano -w /opt/scripts/deletebkp1

Enter the following (using your share name, user, pasword & domain) 

umount /mnt/smb
/bin/mount -t cifs //stationXX/bkp1 /mnt/smb -o username=XXXXXX,password=XXXXXXXXXXXXX
rm -f -R /mnt/smb/servername.yourdomain.com/
rm -f -R /mnt/smb/tmp_dir/

Then save & exit 

Ctrl o
Ctrl x


Create the cron job 

nano -w /etc/cron.d/deletebkp1

Enter the following (setting the time & day to be just before your scheduled backup time), see below. The following values suit a scheduled full backup for 2.00am on Saturday morning (ie Friday evening), where the old backup gets deleted at 1:50am 

50    1   *    *   6    root   . /opt/scripts/deletebkp1

Then save & exit 

Ctrl o
Ctrl x


Cron entries format is as follows: 

+-----------------------Minute            (0-59)
|    +-------------------Hour of Day      (0-23)
|    |    +---------------Day of Month    (1-31)
|    |    |    +-----------Month of Year  (1-12)
|    |    |    |    +-------Day of Week   (0=Sun,6=Sat)
 
50   1    *    *    6    root   . /opt/scripts/deletebkp1

Howto set the backup file slice size

The default slice or part setting is 700Mb (to suit CD's). The slice size can be changed for example to suit 4.7Gb DVD's, by making 3 parts fit on to a DVD, so the required slice size is 1529Mb (allowing for overheads & real data size).

Set the Slice size in the config DB 

config setprop backupwk Slice 1529M

Then expand the template with 

expand-template /etc/dar/DailyBackup.dcf

The next time the backup runs, the resultant backup file will be split into 1.529Gb part sizes.

Adding/Excluding Directories and Files from the backup list

Warning.png Warning:
The e-smith-backup panel is designed to be used as a full disaster recovery backup & restore method for your sme server, and as such relies on the correct directories being included in the backup. Do not delete any of the default directories from 40go-into as these are all essential for a successful full Restore of your backup to a clean install of the sme server operating system (you will need to reinstall add on apps or contribs). You can safely add directories eg /opt (without leading '/' ), as indicated, but that should typically be the extent of changes made.


Adding Files and Directories

The default backup does not include /opt

To add /opt to the backup inclusion list, create a custom template fragment and add opt to the list.

Create custom template fragment to add needed files and directories 

mkdir -p /etc/e-smith/templates-custom/etc/dar/DailyBackup.dcf

nano -w /etc/e-smith/templates-custom/etc/dar/DailyBackup.dcf/41go-into

--go-into opt

Then expand the template 

expand-template /etc/dar/DailyBackup.dcf


Check your backup (eg using the mc dar plugin) to see that /opt is now included. It is also prudent to do a test full restore and verify that /opt (& all other backed up directories & files) are restored, as anticipated. If something is wrong with your backup or restore procedures, then required files may not be backed up and restored, so it is wise to do a test full backup & restore to fully prove your procedures are working correctly.


Important.png Note:
--include <mask> will include single files into the backup when using a path (without leading '/' ) and file name for the mask, or all files matching the mask when using wildcards. Like: 
 --include path/to/file/sampleFileName.txt
 --include "*.backMeUp"

--go-into <path> will include entire directories with all of it's contents (files and subdirs) into the backup. Like: 

 --go-into path/to/dir/sampleDirName


Excluding Files and Directories

As an example let's say you have an ibay called 'mirror' you don't want backed-up. To exclude this ibay from the backup list, create a custom template fragment and exclude it from the the list.

Create custom template fragment to exclude the unneeded files and directories 

 mkdir -p /etc/e-smith/templates-custom/etc/dar/DailyBackup.dcf
 nano -w /etc/e-smith/templates-custom/etc/dar/DailyBackup.dcf/45prune
 
 --prune home/e-smith/files/ibays/mirror/cgi-bin
 --prune home/e-smith/files/ibays/mirror/files
 --prune home/e-smith/files/ibays/mirror/html

Then expand the template 

 expand-template /etc/dar/DailyBackup.dcf

Again, check your backup (eg using the mc dar plugin) to see that the ibay is not backed-up any more.


Important.png Note:
--exclude <mask> will exclude single files when using a path (without leading '/' ) and file name for the mask or all files matching the mask when using wildcards, from the backup. Like: 
 --exclude path/to/file/sampleFileName.txt
 --exclude "*.NoBackup"

--prune <path> will exclude entire directories with all of it's contents (files and subdirs) from the backup. Like: 

 --prune path/to/dir/sampleDirName


Use WOL to power on Workstation

Important.png Note:
This is a new feature and not implimented in SME 8.0. It will be available from SME 8.1


Wake On Lan can be used to power up the backup target workstation before starting the backup.

To use WOL there simply needs to be a new variable added to the backupwk section of the configuration database, nothing else is required. This variable holds the MAC address of the target workstation, if the MAC address is 00:4E:89:F5:FD:2B use: 

db configuration setprop backupwk SmbHostMAC 00:4E:89:F5:FD:2B

There is also the ability to specify how long to wait between the WOL packet being sent and attempting to start the backup process. The default wait time is 300 seconds, this can be varied via another configuration database setting. It is recommended to not set the wait period below 300 seconds. To set the wait time to 600 seconds: 

db configuration setprop backupwk SmbHostDelay 600

The target system must support, and be set up to respond to, the WOL "magic packets". The network infrastructure must also support WOL packets. WOL should work across the local network without problem. WOL packets are not routeable so it won't work across the internet without additional support from hardware and/or software such as a VPN tunnel. Getting WOL to work across the internet is beyond the scope of this documentation but there are plenty of resources available elsewhere. Most wireless connected devices do not work with WOL.

Note that there is no checking that the supplied MAC address is the correct one for the IP address/hostname of the target workstation. MAC address's are effectively static but IP address's/hostnames can change particularly if the target gets its IP address via DHCP. It is quite possible to have the correct target woken up and be ready but the backup to fail because the IP address has changed. However SME's DHCP server very rarely changes the IP address of a given MAC address.

To stop using WOL simply delete the SmbHostMAC variable: 

db configuration delprop backupwk SmbHostMAC

and to be tidy if the optional delay parameter has been set: 

 db configuration delprop backupwk SmbHostDelay

How to start a one-off backup

To backup up to workstation as a one-off, not a scheduled event, run 

/sbin/e-smith/do_backupwk

However note that scheduled backups must be enabled for the above command to work.

Some notes

Some things to keep in mind when using SME-Server backup:

Backup file slices are NOT autonomous

The catalogue of the contents of the backup is put in the last slice. If for some reason the last slice can not be written (e.g. backup medium is full or network failure), is lost or corrupt, the slices already saved are useless.

Naming of the backups on the workstation share or USB disk

  • For a SME server all backups sets are put in a directory which has the FQDN of the server as a name, ie hostname.domain/
  • All backups of a set are put in a subdirectory with the name of the set, ie. hostname.domain/set0/, hostname.domain/set1/, etc.
  • During a backup session, all session files are temporary put in the directory hostname.domain/tmp_dir/
  • In a set the full backup is named full-yyyymmdd
  • In a set the incremental backups are named inc1-yyyymmdd, inc2-yyyymmdd, etc.
  • In a full backup the files (slices) are named full-yyyymmdd.1.dar, full-yyyymmdd.2.dar, etc.
  • In an incremental backup the files (slices) are named incn-yyyymmdd.1.dar, incn-yyyymmdd.2.dar, etc.

All backup files have allways distinct names. If the backup files are moved elsewhere for archiving, you must recreate the directory structure before doing a restore with the server manager.

cifs - mount error 20 = Not a directory

Some network drives don't function correctly with Backup with DAR and the cifs file system. The mount command returns "mount error 20 = Not a directory". See Bug 4923.

This can be worked around by adding the "nounix" option to the mount command in the /etc/e-smith/events/actions/workstation-backup-dar file.


Warning.png Warning:
This change may not be preserved across updates/upgrades to the Backup with Dar sub-system or to SME Server itself.

An update/upgrade may stop your backups working and require this change to be made again.


Edit the above file and find the line: 

$err = qx(/bin/mount -t cifs "$smbhost:$smbshare" $mntdir -o user=$login,pass=$password 2>&1);

and change it to: 

$err = qx(/bin/mount -t cifs "$smbhost:$smbshare" $mntdir -o nounix,user=$login,pass=$password 2>&1);

Save the file.

Manually restoring from dar archive created by e-smith-backup

Restores can be done for:

-the entire system.

-a selected file only.

-a selected directory and all of its contents.

The entire system and a selected file only can be done through the server-manager.

The selected directory can be done via shell. (You will have to adjust for your particular situation.)

First, MOUNT THE BACKUP DRIVE (if required): 

mount /dev/sdc1 /media/BackupDrive1/

You can mount the configured CIFS/SMB destination using

mount -t cifs //$(config getprop backupwk SmbHost)/$(config getprop backupwk SmbShare) $(config getprop backupwk Mount) -o credentials=/etc/dar/CIFScredentials

List available backups

find $(config getprop backupwk Mount)/$(config get SystemName).$(config get DomainName) -name "*.1.dar" | sed s/\.1\.dar$//

Then, RESTORE THE DIRECTORY: 

dar -x /media/BackupDrive1/server.domain.local/set2/full-201408092200 -N -R ./ -w -g home/e-smith/files/ibays/share_data/files

This will restore the directory /home/e-smith/files/ibays/share_data/files to you current directory. If your current directory is /tmp/, the files will be recovered to /tmp/home/e-smith/files/ibays/share_data/files. This will make you able to search through the recovered files and, if needed, move them to their proper location.


Warning.png Warning:
Be sure to use -R ./ (the dot) to restore in the current dir, otherwise you'll restore files/dirs in their original path


You may also have to RESTORE ANY INCREMENTS: 

dar -x /media/BackupDrive1/server.domain.local/set2/inc-001-201408102200.1.dar -N -R ./ -w -g home/e-smith/files/ibays/share_data/files
dar -x /media/BackupDrive1/server.domain.local/set2/inc-002-201408112200.1.dar -N -R ./ -w -g home/e-smith/files/ibays/share_data/files
dar -x /media/BackupDrive1/server.domain.local/set2/inc-003-201408122200.1.dar -N -R ./ -w -g home/e-smith/files/ibays/share_data/files
dar -x /media/BackupDrive1/server.domain.local/set2/inc-004-201408132200.1.dar -N -R ./ -w -g home/e-smith/files/ibays/share_data/files


Important.png Note:
Unmount the backup destination after any manual exploration: 
 umount $(config getprop backupwk Mount)


Retrieved from "https://wiki.koozali.org/index.php?title=Backup_with_dar&oldid=40825"