Difference between revisions of "Affa"

From SME Server
Jump to navigationJump to search
m
 
(208 intermediate revisions by 26 users not shown)
Line 1: Line 1:
{{Languages}}
+
{{Languages|Affa}}
=== Maintainer ===
 
Michael Weinberger
 
  
=== Description ===
+
{{Warning box| 1st Sept 2022 A recent fix to rsync rsync-3.1.2-11.el7_9.x86_64 causes Affa to silently fail. See below.}}
The main purpose of this affa package is to make a SME 7 Server a dedicated backup box in a few minutes. Affa backs up as many SME servers as you like or any other servers which have sshd running and rsync installed. Once it is configured, Affa runs reliably unattended and sends warning messages in case of any errors.  
 
  
All backup archives are full backups, as Affa make use of the hardlink technique. Therefore a new full backup only needs disk space for the differences plus the filesystem overhead for the hardlinks and directories (which is typically 2-3%).
+
{{Level|Advanced}}
 +
 
 +
===Warning===
 +
 
 +
{{Warning box| rsync-3.1.2-11.el7_9.x86_64 causes Affa to silently fail.}}
 +
 
 +
You can see the failure in your logs but nothing beyond that. No files will be backed up after you have installed this update to rsync.
 +
 
 +
See the link to Bug 12165 below.
 +
 
 +
The only current fix is to downgrade rsync on the Affa server. It does not matter on the Target server.
 +
 
 +
We are working on a solution and have test code running and will push a fix as soon as we can.
 +
 
 +
Do please contact us on the bug if you want to help test. A fix will be available faster if you do.
 +
 +
{{usefulnote}}
 +
 
 +
===Maintainer===
 +
Maintainers(s) Affa3: Arnaud, stephdl (and please see above note.)<br>
 +
Copyright (C) 2004-2012 by Michael Weinberger<br>
 +
===Version===
 +
{{#smeversion: smeserver-affa}}
 +
 
 +
===Description===
 +
The main purpose of this affa package is to make a SME Server a dedicated backup box in a few minutes. Affa backs up as many SME servers as you like or any other servers which have sshd running and rsync installed. Once it is configured, Affa runs reliably unattended and sends warning messages in case of any errors.
 +
 
 +
All backup archives are full backups, as Affa makes use of the hardlink technique. Therefore a new full backup only needs disk space for the differences plus the filesystem overhead for the hardlinks and directories (which is typically 2-3%).
  
 
Affa is based on the rsync program and supports the rsync --compress option. This allows you to run backups over the internet or VPN. A typical setup is one or more Affa backup servers placed in different locations, which backup the production server(s) over the VPN.
 
Affa is based on the rsync program and supports the rsync --compress option. This allows you to run backups over the internet or VPN. A typical setup is one or more Affa backup servers placed in different locations, which backup the production server(s) over the VPN.
  
 
A special feature is the rise option, which allows you to rise the backup server to your production server from a backup archive in case of a dead loss of your production server. The rise is executed within a extremely short time, even with huge amount of data. The rise feature uses hardlinks and therefore does not use up additional disk space.
 
A special feature is the rise option, which allows you to rise the backup server to your production server from a backup archive in case of a dead loss of your production server. The rise is executed within a extremely short time, even with huge amount of data. The rise feature uses hardlinks and therefore does not use up additional disk space.
 +
 +
The rise feature can also be used to upgrade from a lower version to a higher version of SME. This also work from SME 8.1 to 9. For this to work you need 2 separate installations of SME. One is the actual running server and the other is an affa backup server with a newer version of SME Server where you execute the rise command. See more here: [[Moving SME to new Hardware]]
 +
 +
A simple way to have a server running with maximum up time is to duplicate the hardware. E.g. to have 2 separate hardware boxes or virtual servers on separate hardware. One is the actual server and the other one is a backup system that in a very short time in case of problems can be upgraded to the actual server with the rise command. This also allow you to upgrade the SME Server software with minimum down time!
  
 
Affa is a command line tool for system administrators and is intentionally designed without a GUI. Therefore it can be efficiently managed on the console and over slow internet connections.
 
Affa is a command line tool for system administrators and is intentionally designed without a GUI. Therefore it can be efficiently managed on the console and over slow internet connections.
  
==== Affa features at a glance ====
+
'''Note:''' This document also refers to the Affa Version 2 which is not maintained anymore. Information on Affa 3 will be gradually added here once it has been more tested.
* Affa is secure: All transfers and connections are made by using the ssh protocol with public/private key authentication
+
 
* Periodically runs unattended full backups. Only differences to the previous full backup are transferred over the network
+
====Affa features at a glance====
* Using rsync with optional bandwidth limit and compression allows backups over the internet
+
 
* Uses hardlink technique, i.e. physical disk space only needed for the differences between two full backups
+
*Affa is secure: All transfers and connections are made by using the ssh protocol with public/private key authentication
* Keeps a configurable number of scheduled, daily, weekly, monthly and yearly full backup archives
+
*Periodically runs unattended full backups. Only differences to the previous full backup are transferred over the network
* The archives are browseable. No need to unpack them first.
+
*Using rsync with optional bandwidth limit and compression allows backups over the internet
* Backup jobs are started by the cron daemon
+
*Uses hardlink technique, i.e. physical disk space only needed for the differences between two full backups
* Interrupted jobs continuing with already transfered data when restarted
+
*Keeps a configurable number of scheduled, daily, weekly, monthly and yearly full backup archives
* Backups the default e-smith directories and files, when property SMEServer is set to yes
+
*The archives are browseable. No need to unpack them first.
* Additional directories and files can be included
+
*Archives can be mapped to a Samba share.
* Directories and files can be excluded from the backup
+
*Backup jobs are started by the cron daemon
* Non-SME server linux systems can be backed up by setting the SMEServer property to no and using an include list
+
*Interrupted jobs continuing with already transfered data when restarted
* Configurable nice level for rsync processes on the backup and source server
+
*Backups the default e-smith directories and files, when property SMEServer is set to yes
* Optional run of custom programs before and after a job run (e.g. running tape backup)
+
*Additional directories and files can be included
* Checks the disk space left after a job run with warning levels strict, normal or risky
+
*Directories and files can be excluded from the backup
* Extensive checking of failure conditions
+
*Non-SME server linux systems can be backed up by setting the SMEServer property to no and using an include list
* Sends failure messages to a configurable list of email addresses
+
*In ESXi mode, running virtual machines can be backed up. See [[Backup of ESXi Virtual Machines using Affa]]
* Sends a warning message, if the backup server runs out of disk space
+
*Configurable nice level for rsync processes on the backup and source server
* Installs an optional watchdog on the source server in case the backupserver fails
+
*Optional run of custom programs before and after a job run (e.g. running tape backup)
* Watchdog sends warning, if an expected backup did not run
+
*Checks the disk space left after a job run with warning levels strict, normal or risky
* Watchdog sends a daily reminder message, if the error continues unchecked
+
*Extensive checking of failure conditions
* Option to display current status of all jobs showing times of last and next run, size and disk usage
+
*Sends failure messages to a configurable list of email addresses
* Status can be mailed on a daily, weekly or monthly schedule
+
*Sends a warning message, if the backup server runs out of disk space
* Option to display all existing archives of a job shown date, size, nbr of files and disk usage
+
*Installs an optional watchdog on the source server in case the backupserver fails (SME Server only)
* Option to send (and revoke) the public key to the source server (SME Server only)
+
*Watchdog sends warning, if an expected backup did not run (SME Server only)
* Option to rise the backup server to a production server from a backup. For SME 7 only
+
*Watchdog sends a daily reminder message, if the error continues unchecked (SME Server only)
* The rise feature does not physically move data and therefore is extremly fast and needs (almost) no extra disk space
+
*Option to display current status of all jobs showing times of last and next run, size and disk usage
* Rise option can be run remotely as the NIC driver configuration of the backup server are preserved
+
*Status can be mailed on a daily, weekly or monthly schedule
* Compares installed RPMs on source with backup server. Sends warning message, if not in sync
+
*Option to display all existing archives of a job shown date, number of files, size and bytes tranferred from the source
* Undo rise option to restore the backup server
+
*Option to send (and revoke) the public key to the source server (SME Server and ESXi only)
* Configurable via a e-smith style db, with one record for each job and a default record for all jobs
+
*Option to rise the backup server to a production server from a backup (SME Server only)
* Logs to /var/log/affa/JOB.log and /var/log/affa/affa.log with optional debug switch for higher verbosity
+
*The rise feature does not physically move data and therefore is extremly fast and needs (almost) no extra disk space
* Log files are rotated weekly, with 5 logs kept
+
*Rise option can be run remotely as the NIC driver configuration of the backup server are preserved
 +
*Compares installed RPMs on source with backup server. Sends warning message, if not in sync
 +
*Undo rise option to restore the backup server
 +
*Configurable via a e-smith style db, with one record for each job and a default record for all jobs
 +
*Logs to /var/log/affa/JOB.log and /var/log/affa/affa.log with optional debug switch for higher verbosity
 +
*Log files are rotated weekly, with 5 logs kept
 +
 
 +
<div class="mw-collapsible mw-collapsed" style="text-align:left" data-expandtext="Obsolete Affa 2 instructions &darr;" data-collapsetext="less &uarr;">{{Warning box|SME v7/8/9 are no longer supported.}}
 +
 
 +
</div>
 +
 
 +
===Installation of Affa 3===
 +
The following instructions assume that a fresh Affa 3 installation is made on a fresh dedicated SME server which serves as a dedicated backup server ('affabox‘). No jobs are migrated from Affa 2. For using an existing SME server, just skip the instructions how to setup a fresh SME box.
 +
 
 +
Setup a dedicated machine with SME 8.1 or SME 9.0 from CDROM. Use the following example settings:
 +
<ol><li>Domain name: athome.xx (use your existing domain name)</li>
 +
<li>Host name: affabox (must not match to existing host)</li>
 +
<li>IP address: 192.168.0.10 (must not match to existing IP address)</li>
 +
<li>Subnet: 255.255.255.0</li>
 +
<li>role: Server only</li>
 +
<li>Gateway: 192.168.0.1 (use your existing gateway)</li>
 +
<li>DHCP: DEactivate</li>
 +
<li>DNS server: 192.168.0.1 (use your existing DNS server)</li>
 +
<li>activate changes: yes</li>
 +
<li>The fresh server will then restart with the above settings.</li>
 +
</ol>
 +
 
 +
To update your server, login as user root on local console. Use 'top' command to display running jobs. Wait until 'yum' stopped running.
 +
yum clean all
 +
yum update
 +
The fresh server will be updated.
 +
signal-event post-upgrade
 +
signal-event reboot
 +
The server will reboot. Afterwards it is ready for installation of Affa 3.
 +
 
 +
Login as root on local or remote SSH console of ''affabox''.
 +
 
  
=== Installation or Update ===
+
/usr/bin/yum install --enablerepo=smecontribs smeserver-affa
+
 
When you have installed Affa for the first time run the following command to initialize the Affa database
+
=For SME10=
  affa --make-cronjobs
+
 
and logout and re-login to the console to take the bash auto-completion (TAB key) in effect.
+
There is a new version 3.3.1
 +
 
 +
However. Affa is a very complicated piece of work with a huge number of options and many have probably not been sufficiently tested depite requests for testing and feedback. Most of the major components should work but there may well be a plethora of edge cases that fail.
 +
 
 +
{{Warning box|Do not rely on this as your sole backup. Make sure you have another type of backup as well}}
 +
 
 +
{{Warning box|Do not use ANY Koozali SME v9 affa versions with v10. It will probably break both your backups and your new server}}
 +
 
 +
We have done very minimal testing on migration from v9 to v10.
 +
 
 +
See https://bugs.contribs.org/show_bug.cgi?id=11024
 +
 
 +
 
 +
Under normal circumstances you would execute this to install :
 +
 
 +
yum install smeserver-extrarepositories-epel smeserver-extrarepositories-openfusion
 +
signal-event yum-modify
 +
config set UnsavedChanges no
 +
 
 +
yum install --enablerepo=smecontribs,epel,openfusion smeserver-affa
 +
 
 +
During development install can be executed with:
 +
yum install --enablerepo=smedev,epel,openfusion smeserver-affa
 +
 
 +
 
 +
 
 +
Please post bug reports in the [http://bugs.contribs.org bug tracker]
 +
 
 +
<headertabs />
 +
 
 +
The server again needs to be updated.
 +
  signal-event post-upgrade
 +
signal-event reboot
 +
The server will reboot. Afterwards Affa 3 is ready for use.
 +
 
 +
{{Warning box| Note that on SME10, a SSH [https://wiki.contribs.org/AutoBlock AutoBlock] feature interferes with the Affa requirement for regular SSH logins! You might have to disable this feature, aor increase the MaxAuthtries to allow Affa to operate correctly.}}
  
{{Note box|After upgrade from Affa versions <nowiki><=0.7.0</nowiki> you need to run the command
+
===Creation of backup jobs===
  affa --send-key
+
Job configurations of Affa 3 are no longer stored in an e-smith style database. They are stored in configuration files located in ''/etc/affa'' . Create your configuration files in this directory, e.g.
(w/o any arguments) to generate the ssh known host entries.}}
+
  touch /etc/affa/backup-smeserver.conf
 +
where ''backup-smeserver'' is your jobname, and add content as described below.
  
=== Quick start example ===
+
====Quick start example====
You have a SME 7 production server with hostname 'prodbox‘ and IP 10.200.48.1.<br>
+
You have a SME production server with hostname 'smeserver‘ and IP 192.168.0.2.<br>
Set up a second SME 7 box as your backupserver with hostname 'affabox‘ and IP 10.200.48.2.  
+
You have a second SME box as your backup server with hostname 'affabox‘ and IP 192.168.0.10.  
  
<ol></li><li>log into the 'affabox' and install the packages as described above.
+
Login to your 'affabox' as root and edit ''/etc/affa/backup-smeserver.conf'' . Using e.g. editor ''nano'', create the following example job configuration file for jobname ''backup-smeserver'' :
</li><li>copy the config helper script sample
+
[backup-smeserver]
cp /usr/lib/affa/jobconfig-sample.pl /root/prodbox-job.pl
+
remoteHostName=192.168.0.2
</li><li>edit /root/prodbox-job.pl and set
+
SMEServer=yes
  my $jobname='prodbox';
+
Watchdog=yes
and
+
RPMCheck=yes
  'remoteHostName‘=>'10.200.48.1',
+
  ConnectionCheckTimeout=120
optionally, if the remote server port is configured to e.g. 2222 then set
+
Debug=no
  'sshPort'=>2222,
+
  Description=Backup of 192.168.0.2 smeserver.athome.xx
</li><li>write the configuration (this makes the database entries and sets up the cronjobs)
+
  DiskSpaceWarn=strict
  /root/prodbox-job.pl
+
RootDir=/var/affa
</li><li>generate the DSA keys and send the public key to the 'prodbox'
+
  TimeSchedule=0630
{{Note box|When initially doing this step, you will need to temporarily enable "Allow secure shell access using standard passwords" on the production server}}
+
localNice=15
  affa --send-key prodbox
+
remoteNice=15
</li><li>run the job manually
+
  rsync--inplace=yes
 +
rsyncCompress=no
 +
rsyncTimeout=900
 +
scheduledKeep=1
 +
dailyKeep=7
 +
weeklyKeep=4
 +
monthlyKeep=12
 +
yearlyKeep=1
 +
status=enabled
  
  affa --run prodbox
+
If you have a Letsencrypt certificate on the production server, then you should also include
</li></ol>
+
  Include=/etc/dehydrated
  
=== Configuration ===
+
Then save your job configuration file.
The configuration is stored in an e-smith style database. Use the db command to configure Affa.
 
The jobname is the record key with the type 'job'.<br>To setup a new job with the name 'prodbox' enter:
 
  
  db affa set prodbox job
+
Now check that your configuration is OK:
 +
  affa --configcheck
 +
This should throw out no errors. Now create / send key files to your productive smeserver:
 +
affa --send-key backup-smeserver
 +
The following output should appear on the console:
 +
Job sme-backup: Generating RSA keys...
 +
Successfully created RSA key pair.
 +
root@192.168.0.2's password: <Enter password of 192.168.0.2/smeserver and hit ENTER>
 +
Public key sent to 192.168.0.2
 +
Now run your job manually, both for test purposes, as well as to create RSA keys:
 +
affa --run backup-smeserver
 +
The following output should appear on the console:
 +
The authenticity of host 'backup-smeserver (192.168.0.2)' can't be established.
 +
RSA key fingerprint is 3b..........65.
 +
Are you sure you want ton continue connecting (yes/no)? <yes>
 +
Now your backup job should run for the first time. Depending on the volume of the files, this takes quite a while.
  
then set the properties
+
Once the job is done, check that the archive is available:
 +
affa –-list-archives
  
  db affa setprop prodbox remoteHostName 192.168.1.1
+
or run the job a second time:
db affa setprop prodbox TimeSchedule '0030,0730,1130,1330,1730,2030'
+
  affa --run backup-smeserver
db affa setprop prodbox Description 'My Production Server'
+
Note that you won't be asked for the password again. Note as well, that this second run of the job ''backup-smeserver'' should run considerably faster, because not all files are copied again: just the difference to the previous run is backed up, the rest is stored as hard links to the existing file copies.
db affa setprop prodbox status enabled
 
and so on...
 
  
Alternatively you can you use a script as described above in the 'Quick start' chapter.
+
Check that the second archive is available too:
 +
affa --list-archives
  
To verify your work, type:
+
From here you are able to work as with Affa 2. Modify your config file as required and described below. Automate the backup using the command ''affa --make-cronjobs'', see below.
  
db affa show prodbox
+
====Job configuration properties====
 +
'''Note 1:''' The default values shown in this table are the Affa program defaults and not to be confused with the preset values in the job configuration files, e.g. backup-smeserver.conf.
  
Finally set up the cronjobs:
+
'''Note 2:''' The complete documentation of the settings of Affa3 is available in [http://affa.sourceforge.net/AffaPdfMan.pdf] . It should be taken in consideration!
affa --make-cronjobs
 
  
==== Job configuration properties ====
+
'''Note 3:''' Affa 3 for SME is a fork of [http://affa.sourceforge.net/ Affa 3 for CentOS]. It adds a few options back again which have been removed. Generally the configuration properties as described [http://wiki.nikoforge.org/Affa_Configuration/ here] do apply. Arnaud added the following functions for SME:
{| border="1" cellpadding="3" cellspacing=0
+
<ul>
 +
<li>The parameter “SMEServer”</li>
 +
<li>The parameter and function “Watchdog”</li>
 +
<li>The parameter and function “RPMCheck”</li>
 +
<li>The functions “--rise” and “--undo-rise”</li>
 +
</ul>
 +
Consequently the list below should reproduce the list for the unforked Affa 3 version, plus adding the properties above. For details refer to [https://www.guedel.eu/index.php/informatique/sme-server-8/affa-v3 Arnaud's website].
 +
{| border="1" cellspacing="0" cellpadding="3"
 
|-
 
|-
| '''Property''' || '''Value''' || '''Default''' || '''Description'''  
+
|'''Property'''||'''Value'''||'''Multivalue'''||'''Default'''||'''Description'''
 
|-
 
|-
| remoteHostName  
+
|remoteHostName
| FQHN or IP || || FQHN or IP of the source host (mandatory)
+
|FQHN or IP
 +
|no
 +
|<none>
 +
|FQHN or IP of the source host (mandatory)
 
|-
 
|-
| TimeSchedule
+
|remoteUser
| HHMM,HHMM,... || || doesn't need to be ordered. At least one time is mandatory
+
|account
 +
|no
 +
|root
 +
|The user account to be used with all ssh logins. May be case sensitive, e.g. Administrator account on Windows
 
|-
 
|-
| Description
+
|Description
| text string || ||
+
|text string
 +
|no
 +
|<none>
 +
|Any text that describes the job
 
|-
 
|-
| scheduledKeep<br>dailyKeep<br>weeklyKeep<br>monthlyKeep<br>yearlyKeep
+
|TimeSchedule
| integer >= 1 || 2<br>7<br>4<br>12<br>2<br>|| how many of the scheduled, daily, weekly, monthly or yearly backups should be kept
+
|HHMM
 +
|yes
 +
|2230
 +
|The time the job will run. Use exactly 4 digits, no colon, no point. '''Important:''' Using the proper format HHMM is essential. Badly formatted TimeSchedule will cause strange Perl errors.  Multiple times can be achieved by repeating the line, each line with a different time.
 
|-
 
|-
| SMEServer
+
|status
| yes ''or'' no || yes || when set to yes the default e-smith directories are automatically included and the property RPMCheck=yes can be used
+
|enabled ''or'' disabled
 
+
|no
 +
|enabled
 +
|When set to disabled, no cron entries will made. You can still run a job manually.
 +
|-
 +
|Include
 +
|full path
 +
|yes
 +
|<none>
 +
|File(s) or directory(s) to be included in the backup.
 +
|-
 +
|Exclude
 +
|full path
 +
|yes
 +
|<none>
 +
|File(s) or directory(s) to be excluded from the backup.
 +
|-
 +
|RootDir
 +
|full path
 +
|no
 +
|/var/affa
 +
|where to store the backup archives. Do not use /home/e-smith or /root as these are included in the backup and therefore the rise option will not work! Recommended: /var/affa
 +
|-
 +
|scheduledKeep
 +
|integer >= 1
 +
|no
 +
|1
 +
|how many of the scheduled backups to be kept
 +
|-
 +
|dailyKeep
 +
|integer >= 0
 +
|no
 +
|7
 +
|how many of the daily backups to be kept
 +
|-
 +
|weeklyKeep
 +
|integer >= 0
 +
|no
 +
|4
 +
|how many of the weekly backups to be kept
 +
|-
 +
|monthlyKeep
 +
|integer >= 0
 +
|no
 +
|12
 +
|how many of the monthly backups to be kept
 +
|-
 +
|yearlyKeep
 +
|integer >= 0
 +
|no
 +
|2
 +
|how many of the yearly backups to be kept
 +
|-
 +
|EmailAddress
 +
|name@domain.com
 +
|yes
 +
|root
 +
|comma separated list of mail addresses, where the messages should be sent to<br>'''Note:''' By default Affa only sends messages on errors, never on success (see property chattyOnSuccess).
 +
|-
 +
|RetryAttempts
 +
|integer >= 0
 +
|no
 +
|4
 +
|When set to a value>0, Affa re-run a failed job RetryAttempts times with a delay of RetryAfter seconds.
 +
|-
 +
|RetryAfter
 +
|seconds >= 0
 +
|no
 +
|900
 +
|when set to a value>0, wait RetryAfter seconds before re-running the job after an error. Only applicable with RetryAttempts>0
 +
|-
 +
|RetryNotification
 +
|yes ''or'' no
 +
|no
 +
|no
 +
|when set to no, Affa does not send an error message when a job has failed and RetryAttempts is configured. An error message is only sent when the last attempt has failed.
 +
|-
 +
|NRPEtrigger
 +
|hours
 +
|no
 +
|24
 +
|NRPE reports a job as critical when the last successful run is older then NRPEtrigger hours. To exclude the job from monitoring set to a value < 0.
 
|-
 
|-
| Include[0]<br>Include[1]<br>...
+
|SambaShare
| full path || || additional files or directories to include
+
|yes ''or'' no
 +
|no
 +
|no
 +
|Access to the job archives via CIFS protocol.
 
|-
 
|-
| Exclude[0]<br>Exclude[1]<br>...
+
|SambaValidUser
| full path || || additional files or directories exclude from backup
+
|local account
 +
|yes
 +
|affa
 +
|User who has permission to access the job archives via the CIFS protocol.
 
|-
 
|-
| RPMCheck
+
|preJobCommand<br> preJobCommandRemote
| yes ''or'' no || no || Only applicable to jobs that backups a SME 7 server.<br>Compares the packages installation of the source host with this affa backup host. Sends a message with diff list if not in sync. This check is usefull, if you want have the option to rise the backup server to a production server from a backup.
+
|path relative to /etc/affa/scripts/
 +
|yes
 +
|<none>
 +
|Scripts to be executed before a job run. The job name and type (scheduled, daily etc.) are passed as arguments to the program. The preJobCommandRemote scripts are copied to the remote server and then executed there. The scripts are executed in alphabetical order. Use a numerical prefix if you need a specific order e.g. 01scriptB, 02remotescriptA, 03scriptA. Use the included prescript-sample.pl and prescriptRemote-sample.pl as a starting point for own scripts.<br>'''Note:''' If you use the parameter “SMEServer=yes”, you may delete the “Included” relative to SME default directories in the jobconfig.ini: they will be backuped automatically.
 
|-
 
|-
| DiskSpaceWarn
+
|postJobCommand<br> postJobCommandRemote
| strict ''or'' normal ''or'' risky ''or'' none || strict || run a disk space check after a job has been completed. With level 'strict' a warning message will be sent, if the available space is less then the size of the just completed backup. With level 'normal'/'risky' the message is sent, if less than 50%/10% of the backup size is still available.
+
|path relative to /etc/affa/scripts/
 +
|yes
 +
|<none>
 +
|Scripts to be executed after a job run. The job name, the type (scheduled, daily etc.) and the exit code of the job run are passed as arguments to the program. The postJobCommandRemote scripts are copied to the remote server and then executed there. The scripts are executed in alphabetical order. Use a numerical prefix if you need a specific order e.g. 01scriptB, 02remotescriptA, 03scriptA. Use the included postscript-sample.pl and postscriptRemote-sample.pl as a starting point for own scripts.<br>'''Note:''' If you use the parameter “SMEServer=yes”, you may delete the “Included” relative to SME default directories in the jobconfig.ini: they will be backuped automatically.
 
|-
 
|-
| localNice
+
|dedup
| -19...+19 || 0 || run rsync local process niced.  
+
|yes ''or'' no
 +
|no
 +
|no
 +
|The purpose of the deduplication is to remove duplicate files to save backup space. When set to 'yes' file deduplication is run after the synchronization has been completed. It looks for files that have identical content, user, group and permissions and replace duplicates by hardlinks. Deduplication scans the just completed archive and the previous one, that usually is
 +
scheduled.0 and daily.0 or scheduled.0 and scheduled.1. Consider this scenario: A user has renamed directories or files. Rsync sees those as new ones and copies them. Deduplication finds the identical copies in the previous archive and replace them by hardlinks. To use deduplication the Freedup program needs to be installed. Affa actually runs freedup -upg scheduled.0 <previous_archive>.
 
|-
 
|-
| remoteNice
+
|dedupKill
| -19...+19 || 0 || run rsync process on source niced.  
+
|yes ''or'' no
 +
|no
 +
|no
 +
|When set to 'no' the job ignores affa --kill or affa --killall when deduplication is running. This is useful in the context of killAt which is typically used to stop bandwith utilisation.
 
|-
 
|-
| Watchdog
+
|DiskSpaceWarn
| yes ''or'' no || yes || Only applicable to jobs that backups a SME 7 server.<br>When a job is started, affa installs a watchdog script on the source in /etc/cron.d/, which sends a warning message, if the next scheduled job (taken from the TimeSchedule property + 10 minutes) did not run. This guarantees, that you will be notfied even in case of a affa server outage. The watchdog script send a daily reminder message, if the error continues. The next run job replaces the watchdog script with a new trigger time.
+
|strict ''or'' normal ''or'' risky ''or'' none
 +
|no
 +
|strict
 +
|Checks disk space remainin on the backup device and issue a warning via email.
 
|-
 
|-
| sshPort
+
|sshPort
| service port || 22 || When sshd on the source host or your firewall listen on a non-standard port set the port here.
+
|service port
 +
|no
 +
|22
 +
|When sshd on the remote host listens on a non-standard port, set the port here.
 
|-
 
|-
| ConnectionCheckTimeout
+
|ConnectionCheckTimeout
| seconds || 120 || before the rsync process is started on the remote source host, affa checks the ssh connection and exits with an error after the configured time, if the host does not respond.
+
|seconds
 +
|no
 +
|120
 +
|Affa checks the ssh connection before the rsync process is started and exits with an error after the configured time if the host did not respond.
 
|-  
 
|-  
| rsyncTimeout
+
|BandwidthLimit
| seconds || 900 || Rsync exits, if no data is transferred for the configured time. This avoids infinitely hanging in case of a network error.
+
|integer>=0 kilobytes per second
 +
|no
 +
|0
 +
|Limits the data transfer rate. A value of zero specifies no limit.
 
|-  
 
|-  
| BandwidthLimit
+
|rsyncTimeout
| integer>=0 kilobytes per second|| 0 || Limits the data transfer rate. A value of zero specifies no limit.
+
|seconds
 +
|no
 +
|900
 +
|Rsync exits after the configured time if no data was transferred. This avoids infinitely hanging in case of a network error.
 
|-  
 
|-  
| rsyncCompress
+
|rsyncCompress
| yes ''or'' no || no || compress the transferred data. May be useful with slow internet connections. Increases CPU load on source and backup host.
+
|yes ''or'' no
 +
|no
 +
|yes
 +
|Compress the transferred data. May be useful with slow internet connections. Increases CPU load on remote and backup host.
 +
|-
 +
|rsync--inplace
 +
|yes ''or'' no
 +
|no
 +
|yes
 +
|Set to no if the rsync version on the remote hist does not support this option.
 
|-
 
|-
| EmailAddresses
+
|rsync--modify-window
| name@domain.com,name@domain.com,... || admin|| comma separated list of mail addresses, where the messages should be sent to<br>'''Note:''' By default Affa only sends messages on errors, never on success (see property chattyOnSuccess).
+
|integer >= 0
 +
|no
 +
|0
 +
|When comparing two timestamps, rsync treats the timestamps as being equal if they differ by no more than the modify-window value. This is normally 0 for an exact match. A value >= 0 is useful if you can't get the clocks of the remote host and the Affa server in sync.
 
|-
 
|-
| chattyOnSuccess
+
|rsyncOptions
| integer >= 0 || 0 || when set to a value>0, Affa sends a message on a successfully completed job run and decrements the chattyOnSuccess value. When the value has reached zero, Affa falls back to the default and only sends messages on errors.
+
|string
 +
|no
 +
|<none>
 +
|Additional option string to be passed to rsync
 
|-
 
|-
| AutomountDevice<br>AutomountPoint
+
|localNice
| full path  || || Device and mountpoint of backup device (e.g. USB disk). Device is automounted before a job starts and unmounted after job completion. With both properties empty no automount is done.
+
| -19...+19
 +
|no
 +
|0
 +
|run rsync local process niced.
 
|-
 
|-
| AutomountOptions
+
|remoteNice
| string  || || An option string passed to the mount command
+
| -19...+19
 +
|no
 +
|0
 +
|run rsync process on source niced.
 
|-
 
|-
| AutoUnmount
+
|killAt
| yes ''or'' no || yes || When set to 'no' the automounted device stay mounted after the Affa run.
+
|HHMM
 +
|no
 +
|<none>
 +
|The time at which a job will be killed if it was still running. You can use it for example to kill jobs that are running over the internet early in the morning so that your users have the full bandwidth available during office hours.
 
|-
 
|-
| preJobCommand<br>postJobCommand
+
|resumeKilledAt
| full path || || programs (local on the affa server) to be executed before/after a job run. The job name and type (scheduled, daily etc.) are passed as arguments to the program. The exit code is additionally passed to the post job command program. See /usr/lib/affa/ for sample perl scripts.
+
|HHMM
 +
|no
 +
|<none>
 +
|The time at which a killed job will be resumed. This allows you to start earlier in the evening than the scheduled time.
 
|-
 
|-
| RootDir
+
|chattyOnSuccess
| full path || /var/affa || where to store the backup archives, Do not use /home/e-smith or /root as these are included in the backup and therefore the rise option will not work! Recommended: /var/affa
+
|integer >= 0
 +
|no
 +
|0
 +
|When set to a value>0, Affa sends a message on a successfully completed job run and decrements the chattyOnSuccess value. When the value has reached zero, Affa falls back to the default and only sends messages on errors.
 
|-
 
|-
| Debug
+
|AutomountDevice<br>AutomountPoint
| yes ''or'' no || no || set to yes to increase log verbosity
+
|full path
 +
|no
 +
|<none>
 +
|Device and mountpoint of backup device (e.g. USB disk). Device is automounted before a job starts and unmounted after job completion. With both properties empty no automount is done.
 
|-
 
|-
| status
+
|AutomountOptions
| enabled ''or'' disabled || enabled || with set to disabled, no cron entries will made. You can still run a job manually.
+
|string
 +
|no
 +
|<none>
 +
|An option string passed to the mount command.
 
|-
 
|-
| rsync--inplace
+
|AutoUnmount
| yes ''or'' no || yes || set to no, if the rsync version on the source does not support this option (like rsync on SME6)
+
|yes ''or'' no
 +
|no
 +
|yes
 +
|When set to 'no' the automounted device stay mounted after the Affa run.
 
|-
 
|-
| rsync--modify-window
+
|Debug
| integer >= 0 || 0 || When comparing two timestamps, rsync treats the timestamps as being equal if they differ by no more than the modify-window value. This is normally 0 for an exact match. A value >= 0 is useful if you can't get the clocks of the source and the Affa server in sync.
+
|yes ''or'' no
 +
|no
 +
|no
 +
|Set to yes to increase log verbosity.
 
|-
 
|-
| rsyncdMode
+
|remoteRsyncBinary
| yes ''or'' no || no || set to yes to connect to the rsync daemon on the remote host (instead of running rsync over ssh)
+
|full path
 +
|no
 +
|/usr/bin/rsync
 +
|If the rsync program on the remote server is located in non-standard location, set it here
 
|-
 
|-
| rsyncdModule
+
|remoteNiceBinary
| string || AFFA || the rsyncd module name (only applicable with rsyncdMode=yes)
+
|full path
 +
|no
 +
|/bin/nice
 +
|If the nice program on the remote server is located in non-standard location, set it here.
 
|-
 
|-
| rsyncdUser
+
|localRsyncBinary
| string || affa || the username for authentication to the rsync daemon (only applicable with rsyncdMode=yes)
+
|full path
 +
|no
 +
|/usr/bin/rsync
 +
|If the local rsync program is located in non-standard location, set it here
 
|-
 
|-
| rsyncdPassword
+
|localNiceBinary
| string || || the password for authentication to the rsync daemon (only applicable with rsyncdMode=yes)
+
|full path
 +
|no
 +
|/bin/nice
 +
|If the nice local program is located in non-standard location, set it here.
 
|-
 
|-
| remoteOS
+
|RemoteAuthorizedKeysFile
| cygwin || || with remoteOS=cygwin the options --send-key and --revoke-key uses the account 'Administrator' and the correct path for the public key on a Windows/Cygwin remote host.
+
|path relative to remote user's home or full path
 +
|no
 +
|.ssh/authorized_keys2
 +
|If the remote host stores the authorized keys file in a non-standard location, set it here.
 +
|-
 +
|rsyncdMode
 +
|yes ''or'' no
 +
|no
 +
|no
 +
|Set to yes to connect to the rsync daemon on the remote host (instead of running rsync over ssh).
 +
|-
 +
|rsyncdModule
 +
|string
 +
|no
 +
|AFFA
 +
|The rsyncd module name (only applicable with rsyncdMode=yes).
 +
|-
 +
|rsyncdUser
 +
|string
 +
|no
 +
|affa
 +
|The username for authentication to the rsync daemon (only applicable with rsyncdMode=yes).
 +
|-
 +
|rsyncdPassword
 +
|string
 +
|no
 +
|<none>
 +
|The password for authentication to the rsync daemon (only applicable with rsyncdMode=yes).
 +
 
 +
|-
 +
|globalStatus
 +
|enabled ''or'' disabled ''or'' jobs
 +
|no
 +
|jobs
 +
|Set to enabled or disabled to overide the status settings in all job sections. When set to value jobs, the status settings in the job sections are effictive.<br>'''Note:''' This property is allowed only in the [GlobalAffaConfig] section.
 +
|-
 +
|sendStatus
 +
|daily ''or'' weekly ''or'' monthly ''or'' never
 +
|no
 +
|weekly
 +
|Defines how frequently the status will be sent.<br>'''Note:''' This property is allowed only in the [GlobalAffaConfig] section.
 +
|-
 +
|SMEServer
 +
|yes ''or'' no
 +
|no
 +
|no
 +
|When set to yes, the default e-smith directories are automatically included and the property RPMCheck=yes can be used.<br>'''Note:''' this property is not implemented in Affa 3 for CentOS. It has been specifically added to the fork for SME.
 +
|-
 +
|RPMCheck
 +
|yes ''or'' no
 +
|no
 +
|no
 +
|Only applicable to jobs that backup a SME server. Compares the packages installation of the source host with this affa backup host. Sends a message with diff list if not in sync. This check is useful, if you want have the option to rise the backup server to a production server from a backup.<br>'''Note:''' this property is not implemented in Affa 3 for CentOS. It has been specifically added to the fork for SME.
 +
|-
 +
|Watchdog
 +
|yes ''or'' no
 +
|no
 +
|yes
 +
|Only applicable to jobs that backups a SME server. When a job is started, affa installs a watchdog script on the source in /etc/cron.d/, which sends a warning message, if the next scheduled job (taken from the TimeSchedule property + 10 minutes) did not run. This guarantees, that you will be notfied even in case of a affa server outage. The watchdog script send a daily reminder message, if the error continues. The next run job replaces the watchdog script with a new trigger time.<br>'''Note:''' this property is not implemented in Affa 3 for CentOS. It has been specifically added to the fork for SME.
 
|}
 
|}
  
==== Default configuration properties ====
+
====Default configuration properties====
All properties can be set as defaults in the DefaultAffaConfig record. This is useful, when you set up many similar jobs.  
+
For all 'name=value' properties defaults can be set in the [GlobalAffaConfig] section which are used in all job configuration when the corresponding property is omitted. For example, when these properties are set in [GlobalAffaConfig] section, they can be omitted in the specific job configurations. This is useful, when you set up many similar jobs.  
Example: You want to set the property 'localNice' to 19 for all jobs. Then run
+
 
  db affa setprop DefaultAffaConfig localNice 19
+
Example: You want to set the property 'localNice' to 19 for all jobs. Then add the following section in one of your configuration files:
and don't set this property for the jobs.
+
[GlobalAffaConfig]
Properties set in the job record overrides the defaults.
+
  localNice=19
 +
Consequently you don't need to set this property for your individual jobs. Properties set in the job record override the defaults.
 +
 
 +
====Properties for global Settings====
 +
The following special properties are only applicable to the [GlobalAffaConfig] section:
 +
<ul>
 +
<li>sendStatus</li>
 +
<li>globalStatus</li>
 +
</ul>
  
The special property 'sendStatus' is only applicable to the DefaultAffaConfig record. It controls the status report sent by email and can be set to the values 'none', 'daily', 'weekly' or 'monthly'. To setup a weekly status report run:
+
All jobs can be disabled for execution through 'cronjobs" with setting 'globalStatus' to 'disabled'.
  db affa setprop DefaultAffaConfig sendStatus weekly
+
  [GlobalAffaConfig]
then setup the cronjob:
+
  globalStatus=disabled
  affa --make-cronjobs
 
  
==== Global disable ====
+
To re-enable run, either delete the above configuration line, or set to:
All jobs can be disabled with setting the AffaGlobalDisable record type to 'yes'.
+
  globalStatus=jobs
  db affa set AffaGlobalDisable yes
+
which is the default value.
affa --make-cronjobs
 
  
to re-enable run:
+
You can also set the value to
  db affa set AffaGlobalDisable no
+
  globalStatus=enabled
affa --make-cronjobs
+
which enforces the the job execution through 'cronjobs' and overrides the specified values in the section for the specific job.
  
=== Usage and command line options ===
+
===Usage and command line options===
 
{{Note box|Options can be abbreviated to uniqueness, e.g. --mak is equal to --make-cronjobs}}
 
{{Note box|Options can be abbreviated to uniqueness, e.g. --mak is equal to --make-cronjobs}}
  affa --run JOB
+
  '''affa --run JOB'''
 
Starts a job run. Usually done by the cronjob.  
 
Starts a job run. Usually done by the cronjob.  
  
  affa --make-cronjobs
+
  '''affa --make-cronjobs'''
Configures the cronjobs as scheduled in the jobs records.
+
Configures the cronjobs as scheduled in the jobs records. Run this command to make changes of time related properties effective i.e. TimeSchedule, killAt and resumeKilledAt properties. By default this command will by executed by 'cronjobs' every 15 minutes. So you don't need to run this command manually, you can also just wait max. 15 minutes before your updates job configurations become effective.
  
  affa --send-key JOB
+
  '''affa --configcheck'''
 +
Checks the syntax and values in all configuration files found in /etc/affa/. Run this command after modifying the configuration. Lower/Upper case errors in property names are corrected automatically.
  
  affa --send-key --host=TARGETHOST [--port=PORT] [--remoteOS=cygwin]
+
  '''affa --send-key [JOB JOB ...]'''
This first generates the DSA key for the Affa Server, if not already done. Then it sends the public key to the host 'remoteHostName' as configured in the record of job JOB and generates the job specific ssh known host entry.  
+
This first generates the RSA key on the Affa Server, if not already done. Then the public key is send to the hosts 'remoteHostName' as defined in section of each job JOB and generates the job specific ssh known host entry.
{{Note box|When initially doing this step, you will need to temporarily enable "Allow secure shell access using standard passwords" on the production server.}}
+
{{Note box|When initially doing this step, you will need to temporarily enable "Allow secure shell access using standard passwords" on the source server.}}
{{Note box|<nowiki>By default, the --send-key option works for a SME Server as a remote server and for systems where the keys are stored in /root/.ssh/authorized_keys2 and the commands /bin/cat, /bin/touch, /bin/grep and /bin/mv are available. With remoteOS=cygwin it works for a Cygwin/Windows remote server.</nowiki>}}
 
  
  affa --full-restore JOB [ARCHIVE]
+
  '''affa --check-connections [JOB JOB ...]'''
Does a full restore of the standard files and directories from the backup ARCHIVE on the remote source server as defined in the JOB record. If ARCHIVE is not given, the archive 'scheduled.0' is used as the default. The full restore reconstructs the server as it was at the time of the backup. After the restore the source host reboots.
+
Checks the ssh login and if applicable rsyncd auth for all jobs given as arguments. Without any arguments all jobs are checked.
  
 +
'''affa --full-restore [--preserve-newer=no] [--delete=yes] JOB [ARCHIVE]'''
 +
Does a full restore remote server of all backed up files and directories from the backup ARCHIVE. If ARCHIVE is not given, the archive 'scheduled.0' is used as the default. The full restore reconstructs the server as it was at the time of the backup. After the restore the source host reboots.
  
affa --rise [--all] JOB [ARCHIVE]
+
With option --preserve-newer=yes files on the remote server with modification time newer than on the backup are not overwritten.
Runs a full restore on the Affa server <b>(!)</b> of all standard files and directories from the backup ARCHIVE of job JOB. In other words: After completion, the Affa box reboots as a clone of the source server. Ensure, that the source server has been powered off before you reboot the Affa box, otherwise the network interface will not come up. This is important, when you run --rise remotely. The --rise feature only works with SME 7 servers und should only be used on dedicated backup servers.
 
  
With option --all, all files and directories of the archive as defined by the include[] properties are restored. Files or directories with the same name on the Affa server will be overwritten and cannot be restored by a undorise. This should not be an issue on a dedicated Affa server which does not hold any other data. After a possible undorise those additional restored dada must be removed manually.
+
With option --delete=yes all files on the remote server, which are not in the backup, are deleted.
  
Please note, that the rise process backs up the the Affa server itself before doing the restore from the archive. This backup is used by a possible undorise run to restore the Affa server. Only the standard files and directories are backed up. Data in non-standard loctions (like /opt) are untouched and will still exist after the rise run. See also: [[Backup_server_config#Standard_backup_.26_restore_inclusions]]
+
If the parameter “SMEServer=yes” is set, the signal-event pre-restore and signal-event post-upgrade will be used automatically.
  
 +
'''affa --list-archives JOB'''
 +
Displays a table of all present archives of job JOB with date, number of files, size and and bytes received. While column buTime shows the actual rsync time, the column ddTime shows the length of the subsequent deduplication run. ddYld is the gained deduplication yield in bytes.
  
affa --create-backup-file JOB [ARCHIVE] [--outfile=FILE]
+
<small><pre>
Creates a gzipped tar archive FILE from the ARCHIVE of job JOB. During creation the MD5 checksum is calculated against which the written tar is checked. The default ARCHIVE is scheduled.0 and default the FILE is ./smeserver.tgz. The checksum is written to FILE.md5sum.
+
Affa version 3.1.0-0 on affa-2.mydomain.com
 +
+------------------------------------------------------------------------------+
 +
| Job: fshare-ak                                                              |
 +
| Description: Fileserver AK                                                  |
 +
| Directory: /var/affa/fshare-ak/                                              |
 +
| Hostname: 10.204.104.4                                                      |
 +
| Email: sysadmin@mydomain.com                                                |
 +
+-----+----------------------+--------+--------+-------+-------+-------+-------+
 +
| Run | Completion date      | buTime | ddTime | ddYld | Files | Size  | Recvd |
 +
+-----+----------------------+--------+--------+-------+-------+-------+-------+
 +
| Y 0 | Wed 2010-06-23 20:26 | 11m53s |      - |    - |  412k |  143G |  470M |
 +
+-----+----------------------+--------+--------+-------+-------+-------+-------+
 +
| M11 | Sun 2010-08-29 20:22 |  7m50s |      - |    - |  417k |  153G |  14M |
 +
| M10 | Sun 2010-09-26 20:23 | 8m57s  |      - |    - |  430k |  156G |  14M |
 +
| M 9 | Sun 2010-10-31 20:25 | 10m05s |      - |    - |  448k |  161G |  15M |
 +
| M 8 | Sun 2010-11-28 20:31 | 16m42s |      - |    - |  463k |  167G |  17M |
 +
| M 7 | Sun 2011-01-02 20:48 | 33m07s |      - |    - |  486k |  173G |  1.1G |
 +
| M 6 | Sun 2011-01-30 20:33 | 18m54s |      - |    - |  493k |  176G |  24M |
 +
| M 5 | Sun 2011-02-27 20:28 | 13m31s |      - |    - |  490k |  176G |  19M |
 +
| M 4 | Sun 2011-03-27 20:28 | 13m08s |      - |    - |  491k |  175G |  17M |
 +
| M 3 | Sun 2011-05-01 20:30 | 15m41s |      - |    - |  493k |  179G |  18M |
 +
| M 2 | Sun 2011-05-29 20:28 | 13m40s |      - |    - |  494k |  182G |  31M |
 +
| M 1 | Sun 2011-06-19 20:26 | 11m20s |      - |    - |  493k |  183G |  17M |
 +
| M 0 | Sun 2011-07-03 20:28 | 13m46s |      - |    - |  496k |  183G |  18M |
 +
+-----+----------------------+--------+--------+-------+-------+-------+-------+
 +
| W 3 | Sun 2011-07-10 20:28 | 13m38s |      - |    - |  479k |  180G |  16M |
 +
| W 2 | Mon 2011-07-11 20:35 | 20m27s |      - |    - |  479k |  180G |  301M |
 +
| W 1 | Fri 2011-07-15 20:30 | 15m53s |      - |    - |  480k |  180G |  62M |
 +
| W 0 | Sun 2011-07-24 19:39 |  9m25s |      - |    - |  482k |  181G |  16M |
 +
+-----+----------------------+--------+--------+-------+-------+-------+-------+
 +
| D 6 | Thu 2011-07-28 19:42 | 12m22s |      - |    - |  483k |  182G |  176M |
 +
| D 5 | Fri 2011-07-29 23:29 | 11m10s |  5h33m |  45G |  483k |  182G |  16M |
 +
| D 4 | Sat 2011-07-30 19:53 | 23m26s |  2h30m |  8.3G |  483k |  182G |  17M |
 +
| D 3 | Sun 2011-07-31 20:07 | 37m31s |  4m47s |  8.3G |  483k |  182G |  17M |
 +
| D 2 | Mon 2011-08-01 20:44 |  1h14m |  7h50m |  8.5G |  484k |  182G |  630M |
 +
| D 1 | Tue 2011-08-02 20:02 | 32m28s | 12h20m |  8.3G |  484k |  182G |  74M |
 +
| D 0 | Wed 2011-08-03 19:58 | 28m46s | 11h01m |  8.5G |  484k |  182G |  214M |
 +
+-----+----------------------+--------+--------+-------+-------+-------+-------+
 +
| S 0 | Thu 2011-08-04 20:00 | 30m28s | 11h52m |  8.5G |  484k |  182G |  203M |
 +
+-----+----------------------+--------+--------+-------+-------+-------+-------+
 +
</pre></small>
  
A smeserver.tgz file stored on an USB disk can be used for restore during a SME Server fresh install.
+
'''affa --list-archives [--csv] JOB'''
 +
With --csv, the output is in machine readable colon separated format.
 +
<small><pre>
 +
Archive:Count;Date;Files;Size;RootDirFilesystemAvail;RootDirFilesystemUsed;valid;TotalBytesReceived;ExecutionTime;DedupTotalFiles;DedupReplacedFiles;DedupSavedBytes;DedupExectime;DedupDate
 +
monthly;00000;201509270631;393237;37252509103;166889260;47876936;yes;1327370;1160638358;99;;;;;
 +
weekly;00003;201510040531;390273;37042612135;164566692;50199504;yes;1117726;45430532;59;;;;;
 +
weekly;00002;201510110531;395553;37299589800;162612204;52153992;yes;1117092;92369237;64;;;;;
 +
weekly;00001;201510180531;403831;37707599172;161855844;52910352;yes;1317083;148317764;64;;;;;
 +
weekly;00000;201510250531;406509;38146200127;161041156;53725040;yes;1104759;64083948;63;;;;;
 +
daily;00006;201510310531;415663;39248862982;162979176;51787020;yes;1958110;198060283;66;;;;;
 +
daily;00005;201511010531;411014;38563389171;163061136;51705060;yes;1156506;48903675;75;;;;;
 +
daily;00004;201511020531;408910;38612857040;162359124;52407072;yes;1007434;210812387;75;;;;;
 +
daily;00003;201511030531;410615;37815861577;162026060;52740136;yes;1524069;162697515;67;;;;;
 +
daily;00002;201511040531;414784;38131734213;161990024;52776172;yes;1571657;134250735;63;;;;;
 +
daily;00001;201511050531;420780;38433895988;161906580;52859616;yes;2021507;149388808;103;;;;;
 +
daily;00000;201511060531;425920;38677614350;161778812;52987384;yes;1987971;166510621;71;;;;;
 +
scheduled;00000;201511070531;425663;38815712018;161612600;53153596;yes;2003540;139603231;68;;;;;
 +
</pre></small>
  
  affa --list-archives [--csv] JOB
+
  '''affa --status [--csv]'''
Displays a table of all present archives of job JOB with date, number of files, size and disk usage. See chapter 'Restore' for an output example.
+
Displays a table of all configured jobs with enable status, time of last and next run, size of the most recent archive, exectution time of the last run and the number of scheduled (S), daily (D), weekly (W), monthly (M) and yearly (Y) archives. If last time shows 'failed', the job did not run in the last 24h. For disabled jobs 'Last' always shows 'failed' after 24 h. To see the date and time of the last run of those jobs use the --list-archives option. Column 'Next' shows the time when the next run will be started, if 'Enabled' is 'yes'. Column 'ddYld' shows the deduplication yield in bytes.
With --csv, the output is in machine readable colon separared format.
 
  
affa --status [--csv]
 
Displays a table of all configured jobs with enable status, time of last and next run, size, disk usage and the number of scheduled (s), daily (d), weekly (w), monthly (m) and yearly (y) archives. Last time shows 'failed', if a job did not run in the last 24h. For disabled jobs 'Last' always shows 'failed' after 24 h. To see the date and time of the last run of those jobs use the --list-archives option. Column 'Next' shows the time when the next run will be started, if 'Enabled' is 'yes'. Column 'Disk usage' shows the usage of the partion, where the RootDir of the job is located on. If all jobs are located in the same RootDir, identical disk usage is shown for all jobs. If RootDir is /var/affa, the usage of the SME server system partition is shown.
 
 
<small><pre>
 
<small><pre>
Affa version 1.0.0-1 on backup.mydomain.de (10.204.48.2)
+
Affa version 3.1.0-0 on affa-2.mydomain.com
+---------------+------+--------+-------+--------+------------+----------------+
+
+--------------+-----+-------+--------+-------+-------+-------+----------------+
| Job           | ENBL | Last  | Next  Size | Disk usage | N of s,d,w,m,y |
+
| Job         | ENA | Last |   Time | Next | Size | ddYld | N of S,D,W,M,Y |
+---------------+------+--------+-------+--------+------------+----------------+
+
+--------------+-----+-------+--------+-------+-------+-------+----------------+
| bookkeep      | yes  | 23:48 | 23:30 |  4.5GB 616GB/47% | 2,7,4,2,0     |
+
| ads-ak      | yes | 20:20 | 0m29s | 20:20 |  28M |    - | 1, 7, 2, 0, 0 |
| crm          | yes | 11:20 | 13:20 47MB 616GB/47% | 7,7,4,2,0      |
+
| ak-user-ma.. | yes | 22:31 | 1m51s | 22:30 |  170M |    - 3, 7, 4, 6, 0 |
| fespdc        | yes  | 09:55 | 12:45 |  40GB 616GB/47% | 6,7,4,2,0      |
+
| azubi-1      | yes | 03:39 | 3h24m | 00:15 44G 1.2G | 1, 7, 4,10, 1 |
| helpdesk      | yes  | failed | 13:40 |   70MB 616GB/47% | 7,7,4,2,0     |
+
| dcpant      | yes | 03:06 | 1m03s | 03:05 | 1.3G |    - | 1, 7, 4,10, 1 |
| imageserv    | yes | 23:01  | 23:00 |   19GB 616GB/47% | 2,7,4,1,0      |
+
| eshare      | yes | 21:06 | 21m01s | 20:45 |  24G 178M | 1, 7, 4,10, 1 |
| intraweb      | yes  | 10:32 | 13:30 |  1.4GB 616GB/47% | 7,7,4,2,0      |
+
| etherpad    | yes | 20:40 | 0m13s | 20:40 | 3.1M |    - 1, 7, 4, 2, 0 |
| pdcaus2      | yes | 23:01 | 12:15 5.4GB 616GB/47% | 2,7,4,2,0     |
+
| fazubi      | yes | 08:16 | 11h16m | 21:00 | 132G 3.3G | 1, 7, 4,10, 1 |
| persoff      | yes | running (pid 17521)     616GB/47% | 2,7,4,2,0     |
+
| fschare-rh  | yes | de-duplicating (pid 9719)              |  1, 7, 4,10, 1 |
| primmail      | yes | 10:09  | 13:00 |  45GB 616GB/47% | 7,7,4,2,0     |
+
| fsh-02      | yes | 07:53 | 12h23m | 19:30 |  182G | 8.5G | 1, 7, 4,10, 1 |
| rayofhope    | yes | 22:32 | 22:30 |   20GB 616GB/47% | 2,7,4,1,0     |
+
| fshare-ak2  | yes | 00:30 | 0m26s | 00:30 |  415M 544k | 1, 7, 4,10, 1 |
| sozserv      | yes | 22:30 | 22:30 8.0GB 616GB/47% | 2,7,4,2,0      |
+
| helpdesk    | yes | 21:27 | 2m16s | 21:25 895M 138k | 1, 7, 4,10, 0 |
+---------------+------+--------+-------+--------+------------+----------------+
+
| it-share    | yes | running rsync (pid 9744)               1, 7, 1, 0, 0 |
| az32share    | no  | -      | 02:20 234MB 616GB/47% | 7,7,2,0,0      |
+
| lightroom    | yes | waiting (pid 9528)                    1, 7, 4, 2, 0 |
| azpdc        | no  | -      | 21:00 42GB 616GB/47% | 2,7,1,0,0      |
+
| localhost    | yes | 02:15 | 0m08s | 02:15 |  395k |     - 1, 7, 2, 0, 0 |
+---------------+------+--------+-------+--------+------------+----------------+
+
| mediawiki    | yes | 16:10 | 0m06s | 19:40 1.1G 6.7M | 3, 7, 4,10, 1 |
 +
| mshare      | yes | 00:48 | 1h33m | 23:15 |  18G 2.4G 1, 7, 4,10, 1 |
 +
| wshare      | yes | 00:34 | 1h49m | 22:45 23G 484M | 1, 7, 4,10, 1 |
 +
+--------------+-----+-------+--------+-------+-------+-------+----------------+
 +
2 disabled jobs not listed. Use --all to display.
 +
</pre></small>
 +
With --csv, the output is in machine readable colon separated format.
  
 +
'''affa --show-config-pathes [--csv] [JOB JOB ...]'''
 +
Prints the full pathes of the file where the section of job JOB is defined.
 +
 +
'''affa --show-default-config'''
 +
Prints a list of all allowed property names with their default values. These values are used, when omitted in the job sections.
 +
 +
'''affa --show-schedule [--all]'''
 +
Prints a 'graphical' timetable for all enabled jobs. The resolution is 30 minutes. An 'S' character marks the scheduled start times. The duration of the job runs are marked with '=' characters.
 +
<small><pre>
 +
Affa version 3.1.0-0 on affa-2.mydomain.com
 +
              TIME 12:00    16:00    20:00    0:00    4:00    8:00
 +
            fsh-02 -------- -------S ~~~~~~~~ ~~~~~~~~ ~~~~~~~~ --------
 +
            ads-ak -------- -------- S------- -------- -------- --------
 +
          etherpad -------- -------- -S------ -------- -------- --------
 +
            eshare -------- -------- -S~----- -------- -------- --------
 +
            fazubi -------- -------- --S~~~~~ ~~~~~~~~ ~~~~~~~~ ~-------
 +
          helpdesk -------- -------- --S----- -------- -------- --------
 +
          it-share -------- -------- ---S=~~~ ~~~~~~~~ ~~~~~~~- -------- busy
 +
            wshare -------- -------- -----S~~ ~~------ -------- --------
 +
ak-user-management -------- -------- -----S-- -------- -------- --------
 +
        fschare-rh -------- -------- -----S=- -------- --~----- -------- busy
 +
            mshare -------- -------- ------S= ~~------ -------- --------
 +
          azubi-1 -------- -------- -------- S=~~~~~~ -------- --------
 +
        fshare-ak2 -------- -------- -------- -S------ -------- --------
 +
        mediawiki -------- -------- -------- ---S---- -------- ----S---
 +
        localhost -------- -------- -------- ----S--- -------- --------
 +
            dcpant -------- -------- -------- ------S- -------- --------
 +
        lightroom -------- -------- -------- -------- -------- S------- busy
 +
Symbols: S=scheduled K=kill R=resume '='=rsync '~'=dedup
 +
2 disabled jobs not listed. Use --all to display.
 
</pre></small>
 
</pre></small>
  
With --csv, the output is printed in a machine readable colon separated format.
+
'''affa --log-tail [JOB]'''
 +
Displays the tail of the logfile of job JOB with live update. This command is identical to tail -n 50 -f /path/to/logfile. Without the JOB argument the global logfile is shown.
  
 +
'''affa --send-status'''
 +
Sends the status table, the disk-usage and the archive list of all jobs to the email addresses configured in the [GlobalAffaConfig] section.
  
  affa --show-schedule
+
  '''affa --disk-usage'''
Prints a 'graphical' timetable for all enabled jobs. The resolution is 30 minutes. Disabled jobs are not shown.
+
Shows the current disk usage of all root dir filesystems
 
<small><pre>
 
<small><pre>
Affa version 1.0.0-1 on backup.mydomain.de (10.204.48.2)
+
Affa version 3.1.0-0 on affa-2.mydomain.de
    TIME 0:00    4:00    8:00    12:00    16:00    20:00 
+
+------+--------+--------+----------------------------------------------------+
      crm -X------ -------- -------- -------- -------- --------
+
| Use% |  Used |  Avail | Root Dir                                          |
  sozserv --X----- -------- -------- -------- -------- --------
+
+------+--------+--------+----------------------------------------------------+
  fespdc ----X--- -------- -------- -------- -------- --------
+
| 71% |  938GB |  365GB | /var/affa                                          |
imageserv -----X-- -------- -------- -------- -------- --------
+
| 20% |  194GB | 759GB | /mnt/affadev                                      |
  primmail ------X- -------- -------- -------- -------- --------
+
+------+--------+--------+----------------------------------------------------+
  intraweb -------- X------- -------- -------- -------- --------
 
  bookkeep -------- ------X- X-X-X-X- X-X-X-X- X-X-X-X- X-X-X---
 
  pdcaus2 -------- -------- -------- X------- -------- ------X-
 
  persoff -------- -------- -------- -------- -------- ---X----
 
helpdesk -------- -------- -------- -------- -------- -----X--
 
rayofhope -------- -------- -------- -------- -------- -----X--
 
2 disabled jobs not listed
 
 
</pre></small>
 
</pre></small>
 +
With --csv, the output is printed in a machine readable colon separated format.
  
 +
'''affa --cleanup JOB'''
 +
After you have lowered a keep value, e.g. scheduledKeep, then archives with a higher indices will no longer be shifted and will exist for ever. This option finds these archives and deletes them after confirmation. When running the cleanup command on a locally attached USB drive, manually mount the USB drive BEFORE running this command. When finished, manually unmount the USB drive. This requirement may apply to some of the other commands listed here.
 +
 +
'''affa --rename-job JOB NEWNAME'''
 +
Renames the job JOB to NEWNAME including the section name and archive directories.
 +
 +
'''affa --move-archive JOB NEWROOTDIR'''
 +
Moves the archive directory of job JOB to the rootdir NEWROOTDIR and adjusts the value of property RootDir. NEWROOTDIR must be a full path starting with a slash. As moving across filesystems (e.g. from an external USB drive to the local disk) is not possible, Affa uses a copy command in this case and deletes the source directory after that. Depending on the archive size, copying across filesystems can take a long time.
 +
 +
'''affa --delete-job [--revoke-key] JOB'''
 +
Irreversibly deletes all archives, of job JOB and set the status property to disable. With --revoke-key option, the public key on the remote server will be deleted.
 +
 +
'''affa --revoke-key JOB'''
 +
Deletes the public RSA key on the remote server.
 +
 +
'''affa --kill JOB'''
 +
Terminates the running job JOB and all its child processes (rsync processes).
  
  affa --send-status
+
  '''affa --killall'''
Sends the status table to the email adresses configured in the 'DefaultAffaConfig' record. Used by the cronjob 'affa-status'.
+
Terminates all running jobs.
  
 
  affa --mailtest JOB
 
  affa --mailtest JOB
Sends a test email to the email adresses configured in the JOB record. With property Watchdog=yes, a test email is sent from the remote host, too. Use this to verify, that your mail processing is functional.<br>
+
Sends a test email to the email addresses configured in the JOB section. Use this to verify, that your mail system is working.
'''Note:''' By default Affa only sends messages on errors, never on success (see property chattyOnSuccess).
+
{{Note box|By default Affa only sends messages on errors, never on success (see property chattyOnSuccess).}}
 +
 
 +
'''affa --nrpe [JOB JOB ...]'''
 +
Checks for jobs that were not run the last NRPRtrigger hours and reports them as failed. A Nagios/ICINGA NRPE complient message is printed and exit status returned.
 +
 
 +
'''affa --version'''
 +
Displays the Affa version number and checks Samba and Freedup installation.
 +
 
 +
'''affa --warranty'''
 +
Displays the disclaimer of liability.
 +
 
 +
'''affa --license'''
 +
Displays the license Affa is released under.
 +
 
 +
'''affa --help'''
 +
Displays a short help.
 +
 
 +
'''affa --debug'''
 +
Enables verbose logging. Overrides job and global configurations.
 +
 
 +
===SME Specific usage and command line options===
 +
 
 +
These commands are implemented in a fork of Affa 3 project for use with SME. '''They are not part of the Affa 3 for CentOS project.'''
 +
 
 +
====SME server 'rise' function====
 +
 
 +
{{Note box|Before trying a rise please check /var/affa/YourServer/rpms-missing.txt
 +
 
 +
You may find there are missing rpms/contribs that template fragments depends on.
 +
 
 +
It may be necessary to install some of these rpms before a rise to prevent issues.
 +
}}
 +
 
 +
The SME server version of Affa has a unique feature restore feature enabling you to turn your Affa backup server into a replica of your server. It is an <b>extremely</b> fast way to restore your server.
 +
 
 +
'''affa --rise [--all] JOB [ARCHIVE]'''
 +
Runs a full restore on the Affa server <b>(!!!)</b> of all standard files and directories from the backup ARCHIVE of job JOB. In other words: After completion, the Affa box reboots as a clone of the source server. Ensure, that the source server has been powered off before you reboot the Affa box, otherwise the network interface will not come up. This is important, when you run --rise remotely. The --rise feature only works with SME servers and should only be used on dedicated backup servers.
 +
 
 +
With option --all, all files and directories of the archive as defined by the include[] properties are restored. Files or directories with the same name on the Affa server will be overwritten and cannot be restored by a --undo-rise. This should not be an issue on a dedicated Affa server which does not hold any other data. After a possible --undo-rise those additional restored data must be removed manually.
 +
 
 +
Please note, that the rise process backs up the the Affa server itself before doing the restore from the archive. This backup is used by a possible --undo-rise run to restore the Affa server. Only the standard files and directories are backed up. Data in non-standard locations (like /opt) are untouched and will still exist after the rise run '''if they don't get overwritten by data of the backup''' (=if the backup contains /opt too e.g.).
  
affa --cleanup JOB
+
{{Note box|The above command is not implemented in Affa 3 for CentOS. It has been specifically added to the fork for SME.}}
After you have lowered a keep value, e.g. scheduledKeep, then archives with a higher indices will no longer be shifted and will exist for ever. This option finds these archives and deletes them.
 
  
  affa --rename-job JOB NEWNAME
+
  '''affa --undo-rise'''
Renames the job JOB to NEWNAME including all database records and archive directories.
+
This feature reverts a risen Affa box to a backup server. After a reboot, all configured jobs '''based on standard files and directories''' will work again.
  
affa --move-archive JOB NEWROOTDIR
+
{{Note box|The above command is not implemented in Affa 3 for CentOS. It has been specifically added to the fork for SME.}}
Moves the archive directory of job JOB to the rootdir NEWROOTDIR and adjusts the property RootDir. NEWROOTDIR must be a full path starting with a slash. As moving across filesystems (e.g. from an external USB drive to the local disk) is not possible, Affa uses a copy command in this case and deletes the source directory after that. Depending on the archive size, copying across filesystems can take a long time.
 
  
affa --delete-job [--revoke-key] JOB
+
===Example setups===
Irreversibly deletes a job including all archives, configuration and report databases. With given --revoke-key option, the public key on the remote server will be deleted.
+
====Dedicated backup server====
  
affa --revoke-key JOB
+
*Setup a dedicated server and install Affa 3 for SME.
 +
*Setup a job for every server you want to backup.
 +
*Send the public keys to every server.
  
  affa --revoke-key --host=TARGETHOST [--port=PORT] [--remoteOS=cygwin]
+
  affa --send-key JOBNAME
Deletes the public dsa key on the remote server.
+
 
{{Note box|<nowiki>By default, the --send-key option works for a SME Server as a remote server and for systems where the keys are stored in /root/.ssh/authorized_keys2. With remoteOS=cygwin it works for a Cygwin/Windows remote server.</nowiki>}}
+
*Check whether password-less logins are working.
  
 
  affa --check-connections
 
  affa --check-connections
Checks the ssh login for all configured jobs. For jobs where the public key was not yet sent, you are prompted for the password and the key will be sent then.
 
  
  affa --kill JOB
+
*Check whether the scheduled jobs are evenly distributed over the day.
Terminates the running job JOB and all its child processes (rsync processes).
+
 
 +
  affa --show-schedule
 +
 
 +
*Create the cron jobs.
  
=== Example setups ===
+
affa --make-cronjobs
==== [Todo] Standard ====
 
Dedicated Affa server to backup all production servers<br>...
 
  
 +
*Check the status after 24 hours.
  
==== [Todo] Local Affa server plus a Affa server in remote location ====
+
affa --status
meanwhile see [http://forums.contribs.org/index.php?topic=36499.msg168352#msg168352 forum post]
 
  
==== Backup single ibays ====
+
====Backup of single ibays====
 
Suppose you want to backup the ibays 'staff1' and 'staff2' on your production server with WAN IP 82.123.1.1  to an Affa server in a different location over the internet every night at 2:30am.
 
Suppose you want to backup the ibays 'staff1' and 'staff2' on your production server with WAN IP 82.123.1.1  to an Affa server in a different location over the internet every night at 2:30am.
<ol></li><li>log into the Affa server and install the packages as described above.
+
 
</li><li>copy the config helper script sample
+
*Log into the Affa server.
cp /usr/lib/affa/jobconfig-sample.pl /root/ibay-staff-job.pl
+
*Create a dedicated job file ''/etc/affa/ibay-staff.conf'' . Edit it and set
</li><li>edit /root/ibay-staff-job.pl and set
+
 
  my $jobname='ibay-staff';
+
[ibay-staff]
and
+
  Description=Backup of ibay-staff on 82.123.1.1
  'remoteHostName‘=>'82.123.1.1',
+
  remoteHostName=82.123.1.1
  'TimeSchedule'=>'0230',
+
  TimeSchedule=0230
  'SMEServer'=>'no',
+
  SMEServer=no
  'Include[0]'=>'/home/e-smith/files/ibays/staff1',
+
  Include=/home/e-smith/files/ibays/staff1
  'Include[1]'=>'/home/e-smith/files/ibays/staff2',
+
  Include=/home/e-smith/files/ibays/staff2
</li><li>write the configuration
+
 
/root/ibay-staff-job.pl
+
*Save the configuration
</li><li>send the public key to the production server
+
*send the public key to the production server
 +
 
 
  affa --send-key ibay-staff
 
  affa --send-key ibay-staff
</li><li>check next morning
+
 
 +
*check next morning
 +
 
 
  affa --list-archives ibay-staff
 
  affa --list-archives ibay-staff
 
  affa --status
 
  affa --status
 
  ls /var/affa/ibay-staff
 
  ls /var/affa/ibay-staff
</li></ol>
 
  
==== Two production servers backup each other ====
+
====Two production servers backup each other====
You have two sites connnected via a VPN and a SME Server running on each site. In that case you don't need a dedicated Affa backup server. Both production servers can additionally act as Affa backup servers backing up the server of the other site. Simply install Affa and configure a job that backs up the other one. You can use all Affa features except of the rise feature.  
+
You have two sites connnected via a VPN and a SME Server running on each site. In that case you don't need a dedicated Affa backup server. Both production servers can additionally act as Affa backup servers backing up the server of the other site. Simply install Affa and configure a job that backs up the other one. You can use all Affa features except of the rise feature.
When using the rise feature the server become any of the backed up systems, which is less useful in this scenario as it would give you a running copy of the server of the other site while the server of this site is down.
+
 
 +
When using the rise feature, the server becomes any of the backed up systems, which is less useful in this scenario as it would give you a running copy of the server of the other site while the server of this site is down.
  
 
To get redundancy and a faster restore you can configure a local backup to an external USB or NAS device.
 
To get redundancy and a faster restore you can configure a local backup to an external USB or NAS device.
Line 386: Line 908:
 
Please pay attention, that you do not backup the archives back to the other site. Set the RootDir property to a path which is not included in the SME default backup list. When leaving the RootDir property to its default '/var/affa', this is guaranteed.
 
Please pay attention, that you do not backup the archives back to the other site. Set the RootDir property to a path which is not included in the SME default backup list. When leaving the RootDir property to its default '/var/affa', this is guaranteed.
  
==== Backing up a Windows computer ====
+
====Use Affa to backup to a NFS-mounted NAS or a local attached USB drive====
Backing up data from a Windows system requires the Cygwin Rsyncd daemon installed and configured on Windows. The standard procedure ''rsync over ssh'' does not work, as the Cygwin rsync process always hangs after some files were transferred.
 
  
===== Rsyncd setup on the Windows computer =====
+
{{Note box|This chapter still needs to be checked whether it works with Affa 3 for SME.}}
Install the Cygwin base, the rsync package and configure the Rsyncd service as described in this document: [[Rsyncd setup on a windows computer for use with Affa backup]]'.
 
The installation of the sshd service is optional and not needed for the backup itself, but having a ssh login can be very helpful for administration or executing scripts on the Windows system. Affa supports sending the public key to a Windows Cygwin for password-less login.
 
  
Note: Affa does not backup the Windows Access Control List (ACL) information. You may need to correct the ACLs manually after a restore.
+
You want to backup your SME production server with hostname 'prodbox‘ and IP 10.200.48.1 on a mounted filesystem instead of setting up a dedicated Affa box.
  
===== Affa Rsyncd mode setup (Quick start example) =====
+
=====Setup NAS=====
You want to backup the ''My Documents'' folders of the users ''ssorglos'' and ''bsimpson'' from the Windows computer 'ws001' with IP 192.168.1.65 to your Affa server 'affabox' with IP 192.168.1.3.
+
You have a [http://www.freenas.org FreeNAS] box with IP 10.200.48.2 up and running with NFS service enabled. The disk is mounted to /mnt/affashare. You have authorized the IP address of your prodbox server to access share /mnt/affashare.
  
1. log into the 'affabox' and copy the Cygwin config helper script sample
+
*log into the 'prodbox' and install the NFS packages
cp /usr/lib/affa/jobconfig-cygwin-sample.pl /root/ws001-mydocs-job.pl
 
2. edit /root/ws001-mydocs-job.pl and set
 
my $jobname='ws001-mydocs';
 
and
 
'remoteHostName‘=>'192.168.1.65',
 
'rsyncdPassword'=>'<i>secretword</i>',
 
'Include[0]'=>'/c/Documents and Settings/ssorglos/My Documents/', ''# don't use backslashes in pathnames!''
 
'Include[1]'=>'/c/Documents and Settings/bsimpson/My Documents/',
 
where ''secretword'' must be replaced by the password you have chosen in the rsyncd.secretsfile on the Windows box.
 
 
 
3. write the configuration (this makes the database entries and sets up the cronjobs)
 
/root/ws001-mydocs-job.pl
 
4. run the job manually. After completion check the archive /var/affa/ws001-mydocs/scheduled.0 and the logfile /var/log/affa/ws001-mydocs.log
 
affa --run ws001-mydocs
 
 
 
===== Affa Rsyncd mode manual setup =====
 
In case you want to do the setup manually using the db command, these are the mandatory settings for Cygwin Rsyncd mode
 
db affa setprop JOB rsyncdMode yes
 
db affa setprop JOB rsyncdModule AFFA
 
db affa setprop JOB rsyncdUser affa
 
db affa setprop JOB rsyncdPassword ''secretword''
 
db affa setprop JOB SMEServer no
 
db affa setprop JOB RPMCheck no
 
db affa setprop JOB Watchdog no
 
Optional for ssh login
 
db affa setprop JOB remoteOS cygwin
 
  
 +
/usr/bin/yum install --enablerepo=smecontribs smeserver-nfs
  
==== Use Affa to backup to a NFS-mounted NAS or a local attached USB drive ====
+
*now enable and start the portmapper service
  
You want to backup your SME 7 production server with hostname 'prodbox‘ and IP 10.200.48.1 on a mounted filesystem instead of setting up a dedicated Affa box.
 
 
===== Setup NAS =====
 
You have a [http://www.freenas.org FreeNAS] box with IP 10.200.48.2 up and running with NFS service enabled. The disk is mounted to /mnt/affashare. You have authorized the IP address of your prodbox server to access share /mnt/affashare.
 
<ol></li><li>log into the 'prodbox' and install the NFS packages
 
/usr/bin/yum install --enablerepo=smecontribs smeserver-nfs
 
Now enable and start the portmapper service
 
 
  config setprop portmap status enabled
 
  config setprop portmap status enabled
 
  service portmap start
 
  service portmap start
  
</li><li>mount the NFS share
+
*mount the NFS share
 +
 
 
  mkdir -p /mnt/affadevice
 
  mkdir -p /mnt/affadevice
 
  mount 10.200.48.2:/mnt/affashare /mnt/affadevice
 
  mount 10.200.48.2:/mnt/affashare /mnt/affadevice
</li></ol>
 
  
===== Alternatively setup a USB drive =====
+
 
<ol></li><li>log into the 'prodbox'  
+
=====Alternatively setup a USB drive=====
</li><li>Connect a USB hard disk to the USB Bus. Now you must determine what device the kernel has assigned to the drive. View the /var/log/message and search for ''Initializing USB Mass Storage driver''. A few lines below you'll find the name of the device. In this example it is ''sdh''. Replace ''/dev/sdh'' by ''your device'' in following instructions.
+
 
</li><li>Use the fdisk program to create a linux partition. '''Verify that this is really the attached USB drive before you continue!'''
+
*log into the 'prodbox'
 +
*connect a USB hard disk to the USB Bus. Now you must determine what device the kernel has assigned to the drive. View the /var/log/message and search for ''Initializing USB Mass Storage driver''. A few lines below you'll find the name of the device. In this example it is ''sdh''. Replace ''/dev/sdh'' by ''your device'' in following instructions.
 +
*use the fdisk program to create a linux partition. '''Verify that this is really the attached USB drive before you continue!'''
  
 
  fdisk /dev/sdh
 
  fdisk /dev/sdh
 
You'll most likely find an existing vfat dos partition, which you have to delete first. In the following we assume, that you have created a single partition ''/dev/sdh1''.
 
You'll most likely find an existing vfat dos partition, which you have to delete first. In the following we assume, that you have created a single partition ''/dev/sdh1''.
</li><li>Now format the drive with an ext3 filesystem
+
 
 +
*now format the drive with an ext3 filesystem
 +
 
 
  mkfs.ext3 /dev/sdh1
 
  mkfs.ext3 /dev/sdh1
</li><li>Make the mount point
+
 
 +
*make the mount point
 +
 
 
  mkdir -p /mnt/affadevice
 
  mkdir -p /mnt/affadevice
</li><li>Add the following line to the /etc/fstab
+
 
 +
*add the following line to the /etc/fstab
 +
 
 
  /dev/sdh1 /mnt/affadevice ext3 defaults
 
  /dev/sdh1 /mnt/affadevice ext3 defaults
</li><li>Mount the drive
+
 
 +
*mount the drive
 +
 
 
  mount /mnt/affadevice
 
  mount /mnt/affadevice
</li><li>Crosscheck your work using the df command
+
 
 +
*crosscheck your work using the df command
 +
 
 
  df
 
  df
</li></ol>
 
  
===== Setup Affa =====
+
=====Setup Affa=====
You want to run backups on 11:30 h, 15:30 h and 19:30 h and you want to keep the last 3 scheduled backups, 7 daily, 5 weekly, 12 monthly and 1 yearly backups.
+
You want to run backups at 11:30 h, 15:30 h and 19:30 h and you want to keep the last 3 scheduled backups, 7 daily, 5 weekly, 12 monthly and 1 yearly backups.
<ol></li><li>log into the 'prodbox' and install the Affa packages as described above.
+
 
</li><li>copy the config helper script sample
+
*log into the 'prodbox' and install the Affa packages as described above.
cp /usr/lib/affa/jobconfig-sample.pl /root/prodbox-job.pl
+
*create/edit the conf file of the job: nano /etc/affa/prodbox.conf
</li><li>edit /root/prodbox-job.pl and set
+
 
  my $jobname='prodbox';
+
and set:
and
+
  [prodbox]
  'remoteHostName‘=>'localhost',
+
  remoteHostName=localhost
  'TimeSchedule'=>'1130,1530,1930',
+
  TimeSchedule=1130
  'scheduledKeep'=>3,
+
TimeSchedule=1530
  'dailyKeep'=>7,
+
TimeSchedule=1930
  'weeklyKeep'=>5,
+
  scheduledKeep=3
  'monthlyKeep'=>12,
+
  dailyKeep=7
  'yearlyKeep'=>1,
+
  weeklyKeep=5
  'RootDir=>'/mnt/affadevice',
+
  monthlyKeep=12
 +
  yearlyKeep=1
 +
  RootDir=/mnt/affadevice
 
Review the other properties and change them to your needs.
 
Review the other properties and change them to your needs.
</li><li>write the configuration
+
 
/root/prodbox-job.pl
+
*run the job manually
</li><li>run the job manually
 
  
 
  affa --run prodbox
 
  affa --run prodbox
</li></ol>
 
  
===== Limitations =====
+
=====Limitations=====
With this kind of setup you cannot use the affa rise feature, as it requires the backup archive to be located on the same fileystem as the server installation. The rise option uses hardlinks, which are not working across filesystems.
+
With this kind of setup you cannot use the affa rise feature, as it requires the backup archive to be located on the '''same fileystem''' as the server installation. The rise option uses hardlinks, which are not working across filesystems.
  
===== Automount =====
+
=====Automount=====
 
Having the backup archives in the same filesystem is always a risk, which can be minimized by using the automount feature. Then the external filesystem is only mounted during a job run.
 
Having the backup archives in the same filesystem is always a risk, which can be minimized by using the automount feature. Then the external filesystem is only mounted during a job run.
  
 
In the NAS example set
 
In the NAS example set
  'AutomountDevice=>'10.200.48.2:/mnt/affashare',
+
  AutomountDevice=10.200.48.2:/mnt/affashare
  'AutomountPoint =>'mnt/affadevice',
+
  AutomountPoint=/mnt/affadevice
 
and skip the step 2.
 
and skip the step 2.
  
 
In the USB drive example set
 
In the USB drive example set
  'AutomountDevice=>'/dev/sdc1',
+
  AutomountDevice=/dev/sdc1
  'AutomountPoint =>'mnt/affadevice',
+
  AutomountPoint=/mnt/affadevice
 
and skip the steps 5 to 8.
 
and skip the steps 5 to 8.
  
 
The mount point will be automatically created, if it does not exist. <br>
 
The mount point will be automatically created, if it does not exist. <br>
For access to the archive directory, you need to mount it manually.
+
To access the archive directory, you need to mount it manually.
 +
 
 +
====Copying a AFFA USB hard drive archive to a new disk====
 +
Affa uses copious amounts of hard links to compress and preserve disk space for its backups. If you are in the situation where you want to copy such a disk archive to a new (bigger) disk, you need to ensure that the hard  links are copied correctly or the destination copy may became significantly bigger than the total of the source archive.
 +
 
 +
One way to copy across file systems (i.e. two different USB disks) and preserve the hard links is as follows:
 +
 
 +
*mount both USB drives but with different mount points. e.g. /media/backup1 & /media/backup2 and then:
 +
 
 +
mkdir /media/backup2/archive
 +
cd /media/backup1/archive
 +
tar cpf - . | ( cd /media/backup2/archive && tar xpf - )
  
=== Restore ===
+
where '''archive''' is the name of the AFFA job you want to move to the other disk.
==== Restore single files or directories ====
 
<b>Example 1:</b>  It's Tuesday April 8th 2008, when user 'briedlin' asks you to restore the messages of his mailbox 'orders' he has inadvertently deleted on Monday morning.
 
  
<ol></li><li>You first must check what backup archives are available. The jobname of this server backup is 'prodserv'.
+
===Restore===
On the Affa backup server do:
+
====Restore single files or directories====
  affa --list-archives prodserv
+
 
<small><pre>
+
'''Example 1:'''  It's Tuesday January 6th 2009, when user 'briedlin' asks you to restore the messages of his mailbox 'orders' he has accidentally deleted on Monday.
Affa version 1.0.0-1 on affa1.mydomain.de (10.204.48.2)
+
 
+------------------------------------------------------------------------------+
+
*You first must check what backup archives are available. The jobname of this server backup is 'primmail'. To get a listing of all archives run
| Job: prodserv                                                                |
+
 
| Description: File- and Mailserver Alfdorf                                    |
+
  affa --list-archives primmail
| Directory: /var/affa/prodserv/                                               |
+
''(see the example listing in chapter [[Affa#Usage_and_command_line_options]]''
| Hostname: 10.204.48.1                                                        |
+
 
| Email: sysadmin@mydomain.de                                                  |
+
*Choose the daily.0 archive, which was created Monday night. Now restore the mailbox 'orders' using the rsync command.
+-----------------------+----------------+--------------+--------+-------------+
+
*Now run the rsync command ''(note the trailing slash!)'' on the Affa backup server:
| Date                  | Archive        |        Files |  Size | Disk usage  |
+
 
+-----------------------+----------------+--------------+--------+-------------+
+
export RDIR=/home/e-smith/files/users/briedlin/Maildir/.orders''# this variable is used to shorten the next command line''
| Sat 2007 Apr 28 00:45 | monthly.10    |        39625 |  7.7GB |    118GB/9% |
+
  rsync -av /var/affa/primmail/daily.0/$RDIR 10.204.48.1:$RDIR
| Sat 2007 May 26 00:41 | monthly.9      |        39928 |  7.8GB |    88GB/6% |
 
| Sat 2007 Jun 30 00:41 | monthly.8      |        40343 |  7.8GB |  155GB/11% |
 
| Fri 2007 Jul 27 00:41 | monthly.7      |        39530 |  7.9GB |  172GB/13% |
 
| Sat 2007 Aug 25 00:41 | monthly.6      |        39613 |  7.9GB |  179GB/13% |
 
| Sat 2007 Sep 29 00:42 | monthly.5      |        40157 |  8.2GB |  198GB/15% |
 
| Sat 2007 Oct 27 00:46 | monthly.4      |        40453 | 7.6GB |  213GB/16% |
 
| Sat 2007 Dec 01 00:43 | monthly.3      |        40545 |  7.6GB |  228GB/17% |
 
| Sun 2007 Dec 30 00:42 | monthly.2      |        40378 |  7.6GB |  284GB/21% |
 
| Sun 2008 Jan 27 00:44 | monthly.1      |        40545 |  7.5GB |  422GB/32% |
 
| Sun 2008 Mar 02 00:42 | monthly.0      |        40924 | 7.7GB |  479GB/36% |
 
+-----------------------+----------------+--------------+--------+-------------+
 
| Sun 2008 Mar 09 00:41 | weekly.3      |        40976 |  7.7GB |  501GB/38% |
 
| Sun 2008 Mar 16 00:42 | weekly.2      |        41027 |  7.7GB |  506GB/38% |
 
| Sun 2008 Mar 23 00:41 | weekly.1      |        41051 |  7.7GB |  500GB/38% |
 
| Sun 2008 Mar 30 00:41 | weekly.0      |        41062 |  7.7GB |  503GB/38% |
 
+-----------------------+----------------+--------------+--------+-------------+
 
| Tue 2008 Apr 01 00:58 | daily.6        |        41090 |  7.7GB |  500GB/38% |
 
| Wed 2008 Apr 02 00:43 | daily.5        |        47680 |  7.8GB |  501GB/38% |
 
| Thu 2008 Apr 03 00:46 | daily.4        |        47680 |  7.8GB |  507GB/38% |
 
| Fri 2008 Apr 04 00:42 | daily.3        |        47686 |  7.8GB |  513GB/39% |
 
| Sat 2008 Apr 05 00:45 | daily.2        |        47687 |  7.8GB |  512GB/39% |
 
| Sun 2008 Apr 06 00:42 | daily.1       |        47687 |  7.8GB |  511GB/39% |
 
| Mon 2008 Apr 07 00:42 | daily.0        |        47687 |  7.8GB |  511GB/39% |
 
+-----------------------+----------------+--------------+--------+-------------+
 
| Tue 2008 Apr 08 00:43 | scheduled.0    |        47688 |  7.8GB |  525GB/40% |
 
+-----------------------+----------------+--------------+--------+-------------+
 
  
</pre></small>
+
If the servers are configured to use a different ssh port eg 2222, then instead do:
  
</li><li>Choose the daily.0 archive, which was created Monday night. Now restore the mailbox 'orders' using the rsync command.
+
  export RDIR=/home/e-smith/files/users/briedlin/Maildir/.orders/  ''# this variable is used to shorten the next command line''
</li><li>Now run the rsync command ''(note the trailing slash!)'' on the Affa backup server:
+
  rsync -av -e 'ssh -i /root/.ssh/id_rsa_affa -p 2222' /var/affa/primmail/daily.0/$RDIR 10.204.48.1:$RDIR
  export RDIR=/home/e-smith/files/users/briedlin/Maildir/.orders/
 
   
 
  rsync -av /var/affa/prodserv/daily.0/$RDIR 10.204.48.1:$RDIR
 
  
</li>If the servers are configured to use a different ssh port eg 2222, then instead do:
 
  
export RDIR=/home/e-smith/files/users/briedlin/Maildir/.orders/
 
rsync -av -e 'ssh -p 2222' /var/affa/prodserv/daily.0/$RDIR 10.204.48.1:$RDIR
 
</li></ol>
 
  
 +
'''Example 2:''' A user has deleted the file orderform.pdf from ibay 'docs' on the server 10.204.48.1 and asks you to restore it.
  
<b>Example 2:</b> A user has deleted the file orderform.pdf from ibay 'docs' and asks you to restore it.
+
*You have searched and found the latest version of this file in weekly archive of job 'prodserv'.
 +
*To copy it back to the server 10.204.48.1 run on the Affa server
  
<ol></li><li>You have searched and found the latest version of this file in archive weekly.
+
  export RFILE=/home/e-smith/files/ibays/docs/files/orderform.pdf ''# this variable is used to shorten the next command line''
</li><li>Copy it back to the server - On the Affa backup server do:
 
  export RFILE=/home/e-smith/files/ibays/docs/files/orderform.pdf
 
 
  rsync -av /var/affa/prodserv/weekly.1/$RFILE 10.204.48.1:$RFILE
 
  rsync -av /var/affa/prodserv/weekly.1/$RFILE 10.204.48.1:$RFILE
  
</li>If the servers are configured to use a different ssh port eg 2222, then instead do:
+
*If the servers are configured to use a different ssh port eg 2222, then instead do:
 +
 
 +
export RFILE=/home/e-smith/files/ibays/docs/files/orderform.pdf  ''# this variable is used to shorten the next command line''
 +
rsync -av -e 'ssh -i /root/.ssh/id_rsa_affa -p 2222' /var/affa/prodserv/weekly.1/$RFILE 10.204.48.1:$RFILE
  
export RFILE=/home/e-smith/files/ibays/docs/files/orderform.pdf
 
rsync -av -e 'ssh -p 2222' /var/affa/prodserv/weekly.1/$RFILE 10.204.48.1:$RFILE
 
</li></ol>
 
  
  
Line 589: Line 1,063:
 
If the servers are configured to use a different ssh port eg 2222, then instead do:
 
If the servers are configured to use a different ssh port eg 2222, then instead do:
  
  rsync -av -e 'ssh -p 2222' "/var/affa/fileshare/daily.4/home/e-smith/files/ibays/mechfiles/files/Valve Control (Design Gr)/VALVE LIST FOR ISSUED.xls" "192.168.1.7:/home/e-smith/files/ibays/mechfiles/files/Valve\\ Control\\ \(Design\\ Gr\)/VALVE\\ LIST\\ FOR\\ ISSUED.xls"
+
  rsync -av -e 'ssh -i /root/.ssh/id_rsa_affa -p 2222' "/var/affa/fileshare/daily.4/home/e-smith/files/ibays/mechfiles/files/Valve Control (Design Gr)/VALVE LIST FOR ISSUED.xls" "192.168.1.7:/home/e-smith/files/ibays/mechfiles/files/Valve\\ Control\\ \(Design\\ Gr\)/VALVE\\ LIST\\ FOR\\ ISSUED.xls"
 +
 
 +
====Full restore====
 +
 
 +
{{Note box|Please for notes about rpms-missing.txt above: https://wiki.contribs.org/Affa#SME_Specific_usage_and_command_line_options}}
 +
 
 +
Generally:
 +
'''affa --full-restore [optional settings] JOB [ARCHIVE]'''
 +
This rsyncs the data (files and directories )from the backup ARCHIVE back to the 'remoteHostname' defined in the configuration of the job JOB.
 +
If ARCHIVE is not given, the archive 'scheduled.0' is used as the default.
 +
The --full-restore reconstructs the server as it was at the time of the backup and takes following IMPORTANT optional settings in consideration:
 +
 
 +
With option [--preserve-newer=yes] files on the remote server with modification time newer than on the backup are not overwritten.
 +
 
 +
With option [--delete=yes] all files on the remote server, which are not in the backup, are deleted.
 +
 
 +
If the parameter “SMEServer=yes” is set, the signal-event pre-restore and signal-event post-upgrade will be used automatically.
  
==== Full restore ====
+
After the restore is done, the restored server reboots automatically.  
To run a full restore of user and configuration data run on the Affa server
 
affa --full-restore <JOB> [<ARCHIVE>]
 
This rsyncs the data from the backup ARCHIVE back to the 'remoteHostname' defined in the configuration of the job JOB.
 
  
 
Example:
 
Example:
You have backuped your production server 'prodsrv' as job 'prodbox'. To restore from the latest backup run
+
You have backuped your production server 'prodsrv' as job 'prodbox'. To restore only lost data from the latest backup run
 
  affa --full-restore prodbox
 
  affa --full-restore prodbox
  
To restore from the older archive daily.3 run  
+
To restore exactly from the older archive daily.3 run  
  affa --full-restore prodbox daily.3
+
  affa --full-restore [--preserve-newer=no] [--delete=yes]  prodbox daily.3
 +
 
 +
{{Warning box|A full restore with --preserve-newer no and --delete yes reconstructs the server as it was at the time of the backup. That means, that all files created or server configuration changes made after the backup will be lost! }}
 +
 
 +
====Moving a SME server installation to new hardware using the Affa rise feature or the backup-restore functions====
 +
Please see this Howto: [[Moving SME to new Hardware]]
 +
 
 +
====Restore from USB drive on new server====
 +
 
 +
This tip comes from http://forums.contribs.org/index.php?topic=42412.0
 +
 
 +
Q) I have complete backups using affa stored on a usb hard drive connected to our affa backup server. I need to restore an earlier monthly backup of our job "mailbackup" to a test server rather than back to the original system.  If I did it from the backup server I see the instructions of how to rise that server to the current backup on that server but I want to restore a point about a month ago before some strange things happened. And I want to do it on a machine that is not our backup server or our production server. I tried to figure out how but am lost in the options. My goal is to do some testing.
  
{{Warning box| A full restore reconstructs the server as it was at the time of the backup. That means, that all files created or server configuration changes made after the backup will be lost. After the restore is done, the restored server reboots automatically. }}
+
A) On your testserver setup a job "mailbackup" identical to that on your backup server (a copy of the conf file is available into the folder of the archive, as an hidden file ''.mailbackup-setup.ini'') but set property '''RootDir''' to /var/affa and property '''status''' to disabled. Connect the USB drive and copy the archive of  the job mailbackup to /var/affa. Then run affa --rise mailbackup ARCHIVE
  
 +
===FAQ===
 +
'''What files and directories are included by default?'''
  
==== Moving a SME 7 server installation to new hardware using the Affa rise feature ====
+
With SMEServer=no nothing at all.
Please see this Howto: [[Moving SME to new Hardware]]
 
  
=== FAQ ===
+
With SMEServer=yes the SME default backup list is the output of following command:
 +
perl -e 'use esmith::Backup;$b=new esmith::Backup;print join("\n",$b->restore_list)."\n"'
 +
and as additional folder:
 +
/etc/affa
  
'''Can I use Affa to backup a SME 6 server?'''<br>
+
{{Note box|The following example needs to be reworked for Affa 3.}}
To backup a SME 6 server set the property 'rsync--inplace' to 'no' and install the perl-TimeDate package on the SME 6 box. The perl-TimeDate package is needed by the watchdog script running on the SME 6. Use the RPM from DAG: [http://dag.wieers.com/rpm/packages/perl-TimeDate/perl-TimeDate-1.16-0.rh73.dag.noarch.rpm perl-TimeDate-1.16-0.rh73.dag.noarch.rpm]<br>
+
'''Can I exclusively backup image files from a specific directory?'''
You cannot use the RPMCheck property.
+
Yes. Assuming you want to backup all gif and jpg files from directory /home/e-smith/files/ibays/pictures/files use this configuration
 +
db affa setprop JOBNAME SMEServer no           # do not include SME Server default directories and files
 +
db affa setprop JOBNAME Include[0] '/home/e-smith/files/ibays/pictures/files' # start searching here
 +
db affa setprop JOBNAME Include[1] '*/'        # search the tree recursively
 +
db affa setprop JOBNAME Include[2] '*.gif'    # copy files that match these patterns
 +
db affa setprop JOBNAME Include[3] '*.jpg' 
 +
db affa setprop JOBNAME Exclude[0] '*'        # exclude all others
 +
db affa setprop JOBNAME rsyncOptions '-m'      # do not create empty folders
  
  
Line 621: Line 1,130:
  
 
'''How do I backup two SME servers behind a firewall?'''
 
'''How do I backup two SME servers behind a firewall?'''
First you need to configure port forwardings for the ssh service on your firewall. Use a non-standard port ,e.g 2200->22, for your second server. Setup a job on your Affa server for each of your production servers. Set up the job property sshPort=2200 for second server.
+
First you need to configure port forwardings for the ssh service on your firewall. Use a non-standard port, e.g 2200->22, for your second server. Setup a job on your Affa server for each of your production servers. Set up the job property sshPort=2200 for second server.
  
=== Uninstall ===
+
===Uninstall===
 
This removes the installed Affa package, all configuration data and all backup archives.
 
This removes the installed Affa package, all configuration data and all backup archives.
  
Line 629: Line 1,138:
 
  affa --revoke-key JOBNAME
 
  affa --revoke-key JOBNAME
 
  affa --delete-job JOBNAME
 
  affa --delete-job JOBNAME
Verify that all jobs have been deleted
+
Verify that all jobs have been deleted:
 
  affa --status
 
  affa --status
Remvove the Affa package
+
Remove the Affa package:
  rpm -e smeserver-affa
+
  yum remove smeserver-affa-3.2.2.1-0.noarch.rpm
and, if perl-Filesys-DiskFree is not needed by any other packages:
+
Cleaning up:
rpm -e perl-Filesys-DiskFree
+
  rm -f /etc/cron.d/affa
Cleaning up
+
  rm -rf /etc/affa
  rm -f /etc/cron.d/affa /etc/cron.d/affa-status
 
  rm -rf /home/e-smith/db/affa /home/e-smith/db/affa-report
 
 
  rm -rf /var/affa /var/log/affa
 
  rm -rf /var/affa /var/log/affa
  
=== Additional information ===
+
===Troubleshooting===
==== Performance ====
+
====Affa stops working after prodserver was rebuild.====  
 +
'''Description:'''
  
It is hard to predict how much time a backup job needs to complete. It depends on the number of files, the total file size, the file changes since last run, the network speed and not least on the CPU power, disk speed and RAM of the source and backup server. The following table of measured values will give you an idea of what you can expect.
+
*Affa stopped working or is broken.
     
+
*When affaserver tries to make an ssh connection to prodserver it fails.
{|  border="1" cellpadding="3" cellspacing=0
+
*In the /var/log/affa/''jobname''.log file you will this message: "SSH connection to ''prodserver ip'' failed. Did you send the public key".
|-
+
*Resending the keys does not solve the problem.
| '''Backup server''' || '''Source server''' || '''Data on source server''' || '''Transferred Data''' || '''Connection''' || '''Compression''' || '''Affa run time'''
 
|-
 
| 2x3.2GHz Xeon<br>2 GB RAM, 1.5 TB RAID6
 
| 2x3.2GHz Dual Core Xeon<br>4 GB RAM, RAID5, SME 7.1<br>Intranet Web Server + MySQL|| 1.4 GB, 12,000 files|| 300 MB, 16 files || Internet 2 Mbit || yes || 2 minutes
 
|-
 
| 2x3.2GHz Xeon<br>2 GB RAM, 1.5 TB RAID6
 
| 2x3.2GHz Dual Core Xeon<br>4 GB RAM, RAID5<br>SME 7.1 Mailserver || 43 GB, 410,000 files || 140 MB, 2,700 files || Internet 2 Mbit|| yes || 10 minutes
 
|-
 
| 2x2GHz Dual Core Xeon 5130<br>6 GB RAM, 1 TB RAID 5
 
| 2x2GHz Dual Core Xeon 5130<br>6 GB RAM, 1 TB RAID 5<br>SME 7.1 File- and Mailserver || 125 GB, 98,000 files || 3,2 GB, 3,000 files || Gbit LAN || no ||  25 minutes
 
|-
 
| 2x2GHz Dual Core Xeon 5130<br>6 GB RAM, 874 GB RAID 5
 
| 2x2GHz Dual Core Xeon 5130<br>6 GB RAM, 1 TB RAID 5<br>SME 7.1 File- and Mailserver || 125 GB, 98,000 files || 3.5 GB, 2000 files || Internet 2 Mbit || yes ||  17 minutes
 
|-
 
| 2x800MHz Pentium 3<br>1 GB RAM, 300 GB RAID1
 
| 2x2.8GHz Xeon,<br> 1GB RAM, 140 GB RAID5<br>SME 7.1 File- and Mailserver || 39 GB, 370,000 files || 12 GB, 4,000 files || 100Mbit LAN || no || 52 minutes
 
|-
 
| 1xP4 2.4GHz<br>256 MB RAM<br>SME 7.1
 
| 2xP4 1GHz<br>1 GB RAM<br>SME 6 || 7.4 GB, 134,790 files || 7.4 GB, 134,790 files || 100Mbit LAN || no || 35 minutes
 
|}
 
  
'''Note:''' The last action of a job run is to remove the oldest backup, e.g. if archive scheduled.11 exists and you have set the scheduledKeep property to 12, then it must be deleted. This can take a significantly long time, which increases the total job execution time.
+
'''Solution:'''
  
A affa --rise run on a SME 7.2, 40 GB data, 411.500 files, 2x2.8 Xeon CPU, 1GB RAM, SW-RAID 5 with 4 U320 SCSI drives took 7 minutes including the final reboot.
+
*This is probably due to the key for prodserver that is stored on the affaserver no longer matches the key of prodserver.
 +
*You need remove the existing key so that the next ssh connection re-gets the keys from prodserver.
 +
*From the command line edit the /root/.ssh/known_hosts file and remove the line for prodserver. It starts with: ''prodserver ip'' ssh-rsa
  
==== Bug report ====
+
===Bug report===
 
Affa is listed in the [http://bugs.contribs.org/enter_bug.cgi?product=SME%20Contribs&component=smeserver-affa bugtracker contribs section].
 
Affa is listed in the [http://bugs.contribs.org/enter_bug.cgi?product=SME%20Contribs&component=smeserver-affa bugtracker contribs section].
Please report all bugs, new feature requests and documentation issues there.  
+
Please report all bugs, new feature requests and documentation issues there.
  
==== Changelog ====
 
<pre>
 
* Wed Apr 23 2008 Michael Weinberger 1.0.0-2
 
  Bug fix #4210, command "affa --show-schedule" executes postJobCommand
 
* Fri Apr 04 2008 Michael Weinberger 1.0.0-1
 
  Final release
 
  man page updated
 
* Fri Apr 04 2008 Michael Weinberger 0.10.0-2
 
  added --all option to --rise
 
* Mon Mar 10 2008 Michael Weinberger 0.10.0-1
 
  Increased release number
 
  Added 10 sec timeout in Net::DNS::Resolver->search
 
* Mon Mar 10 2008 Michael Weinberger 0.9.0-11
 
  Specfile fix
 
* Mon Mar 10 2008 Michael Weinberger 0.9.0-10
 
  Delete locally created watchdog file after remotecopy
 
  Bugfix 4035
 
  Added default AutoUnmount property to sample script
 
  Fix man page
 
  Do not delete md5sum in --create-backup-file when verify is skipped by user
 
  Set lock in --create-backup-file
 
* Sun Mar 09 2008 Michael Weinberger 0.9.0-9
 
  trap signals in --create-backup-file
 
* Sat Mar 08 2008 Michael Weinberger 0.9.0-8
 
  Permissions in /usr/lib/affa
 
* Sat Mar 08 2008 Michael Weinberger 0.9.0-7
 
  md5sum in --create-backup-file
 
* Fri Mar 07 2008 Michael Weinberger 0.9.0-6
 
  Added option --create-backup-file
 
  Using line number as error code in affaErrorExit()
 
  Fixed /etc/profile.d/affa.sh
 
* Mon Mar 03 2008 Michael Weinberger 0.9.0-5
 
  delete new style Dovecot's index files dovecot.index*
 
  delete dovecot index files on remote server in --full-restore
 
* Fri Feb 29 2008 Michael Weinberger 0.9.0-4
 
  set localhost as default remoteHostName in /etc/e-smith/templates//etc/cron.d/affa/00jobs
 
* Wed Feb 27 2008 Michael Weinberger 0.9.0-3
 
  removed %post from spec file
 
* Sat Feb 02 2008 Michael Weinberger 0.9.0-2
 
  Bugfix: Report db was not deleted in deleteJob()
 
  don't use -q on ssh commands with Debug=yes
 
* Fri Feb 1 2008 Shad L. Lords <slords@mail.com> 0.9.0-1
 
  Fix specfile for building in new buildsystem
 
* Fri Jan 18 2008 Michael Weinberger
 
  Version 0.9.0
 
  Bugfix: run cronSetup() after job delete
 
  Throw error if nothing was backed up (empty archive)
 
  Added rsyncd support for backing up Windows/Cygwin
 
  Added option and property remoteOS. Which remoteOS=cgywin the
 
  send key and revoke key functions use the right path to the
 
  authorized_keys2 file and login as user Administrator
 
* Sat Dec 29 2007 Michael Weinberger
 
  Version 0.8.1
 
  Automount Bugfix [Bug 3165]: Only exit with error on mount failure when in backup run. Otherwise log the error and  continue.
 
* Thu Dec 20 2007 Michael Weinberger
 
  Version 0.8.0
 
  added bash completion script
 
  added option --kill JOB
 
  improved INT and TERM signal handling
 
  replaced all scp commands by rsync
 
  added poperty BandwidthLimit (unit KBytes/sec)
 
  each job write its own known hosts entry using ssh option HostKeyAliasOption
 
  affa --send-key w/o arguments sends the keys for all jobs
 
  removed Property StrictHostKeyChecking
 
  UPDATE FROM VERSIONS<8: run the command 'affa --send-key' to generate the known hosts entries for all jobs
 
* Wed Dec 12 2007 Michael Weinberger
 
  Version 0.7.0
 
  Modified logging: Job run logs to /var/log/affa/JOBNAME.log, all other to /var/log/affa/affa.log
 
  Bugfix: Check error status of --send-key  and --revoke-key
 
  New: Options --show-schedule
 
  Modified: Option --list-archives now accepts multiple args or none
 
  Fix: install watchdog on source server before every run not only scheduled, to avoid watchdog triggered if a non-scheduled run take to long time.
 
* Wed Nov 28 2007 Michael Weinberger
 
  Version 0.6.8
 
  Bugfix: Automount was done too late. Daily backup never ran.
 
* Wed Nov 21 2007 Michael Weinberger
 
  Version 0.6.7
 
  Bugfix: Size and No. of files were not shown in --status
 
  Improved deleting of directories
 
  added --check-connections option
 
  Increased width of job column in --stats table
 
* Sun Nov 11 2007 Michael Weinberger
 
  Version 0.6.6
 
  Property rsync--modify-window added
 
  Modified recursivly deleting directories
 
* Wed Nov 07 2007 Michael Weinberger
 
  Version 0.6.5
 
  ssh options added to scp commands
 
  added post-backup event
 
* Wed Aug 08 2007 Michael Weinberger
 
  Version 0.6.4
 
  delete dovcot index files after a --rise
 
* Thu Jul 05 2007 Michael Weinberger
 
  Version 0.6.3
 
  Bugfix: with RPMCheck=yes the affa-rpmlist was not found in the
 
  archive. Wrong path to schedule.0 instead of scheduled.running.
 
  Side effect of changes in version 0.5.0. [Bug 3128]
 
* Tue Jul 03 2007 Michael Weinberger
 
  Version 0.6.2
 
  Bugfix: Filesys::DiskFree requires $ENV{LANG}="en_US" [Bug 3119]
 
  Bug reported to bug-Filesys-DiskFree [at] rt.cpan.org
 
* Mon Jun 25 2007 Michael Weinberger
 
  Version 0.6.1
 
  Fix 3080 was not applied.
 
* Mon Jun 18 2007 Michael Weinberger
 
  Version 0.6.0
 
  Replaced Filesys::DiskSpace by Filesys::DiskFree as the latter supports CIFS file system
 
  using Filesys::DiskFree->mount and ->device instead of scanning /proc/mounts
 
  mkdir RootDir only on job run to avoid making it in an unmounted mountpoint
 
* Mon Jun 18 2007 Michael Weinberger
 
  Version 0.5.2
 
  Sending ssh key failed. When grep returned an error
 
  the mv command was not executed. Using ';' instead of '&&'
 
  for chaining commands. [Bug 3080]
 
  /root/.ssh/authorized_keys2: No such file or directory warning.
 
  Run a touch to guarantee, that the file exists. [Bug 3080]
 
  same issue with --revoke-key
 
* Thu Jun 14 2007 Michael Weinberger
 
  Version 0.5.1
 
  Bugfix automount: getStatus() unmounts the device while a job is running
 
  Status table: Last=never, if there are no archives at all
 
* Tue Jun 12 2007 Michael Weinberger
 
  Version 0.5.0
 
  New property AutomountOptions
 
  status table: grouped disabled jobs. Show Last='-' for disabled jobs instead of 'failed'
 
  unlink lockfile in getLock()
 
  Changes to improve robustness in case of network outage:
 
  - writing to a temporary archive scheduled.running and linkdest
 
    against the latest existing archive (usually scheduled.0). Then
 
    shift archives and rename scheduled.running to scheduled.0
 
  - scheduledKeep=1 is now allowed
 
  - also check connection in daily,weekly etc., not only in scheduled run
 
  - don't run daily, weekly, monthly or yearly, when scheduled.0 doesn't exist.
 
  new option --revoke-key
 
  added optional option --revoke-key to --delete-job
 
  avoid multiple adding public key to authorized_keys2 on source server
 
* Wed Jun 06 2007 Michael Weinberger
 
  Version 0.4.7
 
  use StrictHostKeyChecking=no in sendKeys()
 
* Tue Jun 05 2007 Michael Weinberger
 
  Version 0.4.6
 
  mailtest: Send also a testmail from the remotehost, when property Watchdog=yes
 
  removed useless code in mailTestWatchdogRemote()
 
* Mon Jun 04 2007 Michael Weinberger
 
  Version 0.4.5
 
  write job config file to the archive dir
 
* Thu May 31 2007 Michael Weinberger
 
  Version 0.4.4
 
  added property StrictHostKeyChecking (ssh option)
 
  string comparison instead of numric for $job{'doneYearly'} ne $thisYear etc.
 
* Wed May 30 2007 Michael Weinberger
 
  Version 0.4.3
 
  Minor bugfixes:
 
  calculation of RootDirFilesystemUsage in .AFFA-REPORT
 
  improved error handling with rsync status and df in DiskspaceWarn()
 
* Tue May 29 2007 Michael Weinberger
 
  Version 0.4.2
 
  Infinite loop in execPostJobCommand() if command could not be executed:
 
  Don't call execPostJobCommand() in affaErrorExit() if err==115
 
  RPMCheck property was ignored
 
  added Property chattyOnSuccess
 
  modified jobconfig-sample.pl to preserve 'doneDaily','doneWeekly','doneMonthly','doneYearly' and 'chattyOnSuccess'
 
  write error codes of affaErrorExit() to log
 
* Thu May 24 2007 Michael Weinberger
 
  Version 0.4.1
 
  fixed bug in disk usage calculation
 
* Mon May 21 2007 Michael Weinberger
 
  Version 0.4.0
 
  added option --rename-job
 
  changed syntax of --send-keys (!)
 
  added option --move-archive
 
* Sun May 20 2007 Michael Weinberger
 
  Version 0.3.3
 
  rebuild cronjobs after rise/undorise
 
  ignore job of own backup, when creating cronjobs (job appears after a rise run)
 
* Sun May 20 2007 Michael Weinberger
 
  Version 0.3.2
 
  man: added sshPort propperty
 
  chdir /tmp to avoid cwd warnings when the cwd disappears while running rise or undo rise
 
  bugfix: undorise() did not found own backup archive. Was searching for a wrong name
 
* Wed May 16 2007 Michael Weinberger
 
  Version 0.3.1 minor bugfixes
 
  checkCrossFS() did not work (used in --rise)
 
  jobconfig-sample.pl: deleting record before setting props
 
  Perl errors with --status before a job run
 
* Thu May 10 2007 Michael Weinberger
 
  Version 0.3.0
 
  man page completed
 
  mark archives with indices > keep setting with '*' in --list-acrchive output
 
  Option --delete-job
 
  Option --cleanup
 
  added --job=JOB alternative to --send-keys
 
* Wed May 09 2007 Michael Weinberger
 
  Version 0.2.0
 
  added --mailtest option
 
* Tue May 08 2007 Michael Weinberger
 
  Version 0.1.5
 
  improved --status output
 
  removed options --report and --send-report
 
  added option --show-archives
 
  added --csv for status and show-archives output in CSV format
 
  added property 'sshPort'
 
* Mon May 07 2007 Michael Weinberger
 
  Version 0.1.4
 
  don't install the remote watchdog, when remotehost is eq localhost
 
  improved check for remoteHostName eq localhost using DNS
 
  ssh -o PasswordAuthentication=no in checkConnection()
 
  added --full-restore
 
  missing check for HOSTNAME argument in --send-keys added
 
  improved error check
 
  prevent run of --rise of localhost from own backup
 
* Mon Apr 30 2007 Michael Weinberger
 
  Version 0.1.3
 
  modified 'use constant* syntax in watchdog script for compatibility with perl 5.6 on SME6
 
* Fri Apr 27 2007 Michael Weinberger
 
  Version 0.1.2
 
  Bugfix: Preserve of ethernet driver setting with --rise did not work.
 
  Also preserve NIC bonding.
 
* Mon Apr 23 2007 Michael Weinberger
 
  Version 0.1.1
 
  scheduledKeep must be>=2 for --link-dest
 
  set scheduledKeep to 2 if <2
 
  get lastrun date from affa-report rather than from report file
 
  added auto mount function
 
  added AutomountDevice and AutomountPoint to jobconfig-sample.pl
 
* Wed Apr 18 2007 Michael Weinberger
 
  Version 0.0.8
 
  dont die if report db does not exist
 
* Wed Apr 18 2007 Michael Weinberger
 
  Version 0.0.7
 
  run checkConnection() only for scheduled backups
 
  added Size and Disk usage information to --status
 
* Thu Apr 12 2007 Michael Weinberger
 
  Version 0.0.5
 
  fixed calculation of lastrun-now
 
* Thu Apr 12 2007 Michael Weinberger
 
  Version 0.0.4
 
  added --send-status plus templates
 
  fixed format error of times in affa --status
 
  show 'failed', if lastrun is older 1 day in affa --status
 
  fixed typo. default status=disabled (was disable)
 
* Fri Apr 06 2007 Michael Weinberger
 
  Version 0.0.3
 
  watchdog reminder was not deleted on soure
 
  wrong version mismatch list in  rpm compare
 
* Thu Apr 05 2007 Michael Weinberger
 
  added 'rsync--inplace' property
 
* Mon Apr 02 2007 Michael Weinberger
 
  initial release
 
</pre>
 
  
==== Files ====
+
{{#bugzilla:columns=id,product,version,status,summary |sort=id |order=desc |disablecache=1|component=smeserver-affa|noresultsmessage="No open bugs found."}}
<pre>
 
/etc/e-smith/events/actions/affa-make-cronjobs
 
/etc/e-smith/events/post-upgrade/S90affa-make-cronjobs
 
/etc/e-smith/templates/etc/cron.d/affa-status/00run
 
/etc/e-smith/templates/etc/cron.d/affa/00jobs
 
/etc/logrotate.d/affa
 
/etc/profile.d/affa.sh
 
/sbin/e-smith/affa
 
/sbin/e-smith/affa-rpmlist.sh
 
/usr/lib/affa/COPYING
 
/usr/lib/affa/jobconfig-cygwin-sample.pl
 
/usr/lib/affa/jobconfig-sample.pl
 
/usr/lib/affa/postJobCommand-sample.pl
 
/usr/lib/affa/preJobCommand-sample.pl
 
/usr/lib/affa/watchdog-mailtest.template
 
/usr/lib/affa/watchdog.template
 
/usr/man/man1/affa.1.gz
 
</pre>
 
  
 +
===Additional information===
 +
For details of performance, changelog etc see [[:Affa:Additional information]]
  
==== Acronym ====
+
===References===
Affa stands for '''A'''utomatische '''F'''estplatten '''F'''ern'''a'''rchivierung
+
 
 +
*Wikipedia article http://en.wikipedia.org/wiki/Rsync
 +
*Rsync homepage http://rsync.samba.org/
 +
*Wiki article [[Moving SME to new Hardware]]
 +
*Wiki article [[Backup with Affa and FreeDup]]
 +
*Wiki article [[Rsyncd setup on a windows computer for use with Affa backup]]
 +
*Wiki article [[Backup Windows Computers Using Affa]]
 +
*Wiki article [[Backup of ESXi Virtual Machines using Affa]]
 +
 
 +
----
 
[[Category: Contrib]]
 
[[Category: Contrib]]
 
[[Category: Backup]]
 
[[Category: Backup]]
----
 

Latest revision as of 17:25, 11 September 2022



Warning.png Warning:
1st Sept 2022 A recent fix to rsync rsync-3.1.2-11.el7_9.x86_64 causes Affa to silently fail. See below.



PythonIcon.png Skill level: Advanced
The instructions on this page may require deviations from standard procedures. A good understanding of linux and Koozali SME Server is recommended.


Warning

Warning.png Warning:
rsync-3.1.2-11.el7_9.x86_64 causes Affa to silently fail.


You can see the failure in your logs but nothing beyond that. No files will be backed up after you have installed this update to rsync.

See the link to Bug 12165 below.

The only current fix is to downgrade rsync on the Affa server. It does not matter on the Target server.

We are working on a solution and have test code running and will push a fix as soon as we can.

Do please contact us on the bug if you want to help test. A fix will be available faster if you do.

Is this article helpful to you?
Please consider donating or volunteering
Thank you!

Maintainer

Maintainers(s) Affa3: Arnaud, stephdl (and please see above note.)
Copyright (C) 2004-2012 by Michael Weinberger

Version

Devel 10:
Contrib 10:
Contrib 9:
smeserver-affa
The latest version of smeserver-affa is available in the SME repository, click on the version number(s) for more information.


Description

The main purpose of this affa package is to make a SME Server a dedicated backup box in a few minutes. Affa backs up as many SME servers as you like or any other servers which have sshd running and rsync installed. Once it is configured, Affa runs reliably unattended and sends warning messages in case of any errors.

All backup archives are full backups, as Affa makes use of the hardlink technique. Therefore a new full backup only needs disk space for the differences plus the filesystem overhead for the hardlinks and directories (which is typically 2-3%).

Affa is based on the rsync program and supports the rsync --compress option. This allows you to run backups over the internet or VPN. A typical setup is one or more Affa backup servers placed in different locations, which backup the production server(s) over the VPN.

A special feature is the rise option, which allows you to rise the backup server to your production server from a backup archive in case of a dead loss of your production server. The rise is executed within a extremely short time, even with huge amount of data. The rise feature uses hardlinks and therefore does not use up additional disk space.

The rise feature can also be used to upgrade from a lower version to a higher version of SME. This also work from SME 8.1 to 9. For this to work you need 2 separate installations of SME. One is the actual running server and the other is an affa backup server with a newer version of SME Server where you execute the rise command. See more here: Moving SME to new Hardware

A simple way to have a server running with maximum up time is to duplicate the hardware. E.g. to have 2 separate hardware boxes or virtual servers on separate hardware. One is the actual server and the other one is a backup system that in a very short time in case of problems can be upgraded to the actual server with the rise command. This also allow you to upgrade the SME Server software with minimum down time!

Affa is a command line tool for system administrators and is intentionally designed without a GUI. Therefore it can be efficiently managed on the console and over slow internet connections.

Note: This document also refers to the Affa Version 2 which is not maintained anymore. Information on Affa 3 will be gradually added here once it has been more tested.

Affa features at a glance

  • Affa is secure: All transfers and connections are made by using the ssh protocol with public/private key authentication
  • Periodically runs unattended full backups. Only differences to the previous full backup are transferred over the network
  • Using rsync with optional bandwidth limit and compression allows backups over the internet
  • Uses hardlink technique, i.e. physical disk space only needed for the differences between two full backups
  • Keeps a configurable number of scheduled, daily, weekly, monthly and yearly full backup archives
  • The archives are browseable. No need to unpack them first.
  • Archives can be mapped to a Samba share.
  • Backup jobs are started by the cron daemon
  • Interrupted jobs continuing with already transfered data when restarted
  • Backups the default e-smith directories and files, when property SMEServer is set to yes
  • Additional directories and files can be included
  • Directories and files can be excluded from the backup
  • Non-SME server linux systems can be backed up by setting the SMEServer property to no and using an include list
  • In ESXi mode, running virtual machines can be backed up. See Backup of ESXi Virtual Machines using Affa
  • Configurable nice level for rsync processes on the backup and source server
  • Optional run of custom programs before and after a job run (e.g. running tape backup)
  • Checks the disk space left after a job run with warning levels strict, normal or risky
  • Extensive checking of failure conditions
  • Sends failure messages to a configurable list of email addresses
  • Sends a warning message, if the backup server runs out of disk space
  • Installs an optional watchdog on the source server in case the backupserver fails (SME Server only)
  • Watchdog sends warning, if an expected backup did not run (SME Server only)
  • Watchdog sends a daily reminder message, if the error continues unchecked (SME Server only)
  • Option to display current status of all jobs showing times of last and next run, size and disk usage
  • Status can be mailed on a daily, weekly or monthly schedule
  • Option to display all existing archives of a job shown date, number of files, size and bytes tranferred from the source
  • Option to send (and revoke) the public key to the source server (SME Server and ESXi only)
  • Option to rise the backup server to a production server from a backup (SME Server only)
  • The rise feature does not physically move data and therefore is extremly fast and needs (almost) no extra disk space
  • Rise option can be run remotely as the NIC driver configuration of the backup server are preserved
  • Compares installed RPMs on source with backup server. Sends warning message, if not in sync
  • Undo rise option to restore the backup server
  • Configurable via a e-smith style db, with one record for each job and a default record for all jobs
  • Logs to /var/log/affa/JOB.log and /var/log/affa/affa.log with optional debug switch for higher verbosity
  • Log files are rotated weekly, with 5 logs kept
Warning.png Warning:
SME v7/8/9 are no longer supported.

Installation of Affa 3

The following instructions assume that a fresh Affa 3 installation is made on a fresh dedicated SME server which serves as a dedicated backup server ('affabox‘). No jobs are migrated from Affa 2. For using an existing SME server, just skip the instructions how to setup a fresh SME box.

Setup a dedicated machine with SME 8.1 or SME 9.0 from CDROM. Use the following example settings:

  1. Domain name: athome.xx (use your existing domain name)
  2. Host name: affabox (must not match to existing host)
  3. IP address: 192.168.0.10 (must not match to existing IP address)
  4. Subnet: 255.255.255.0
  5. role: Server only
  6. Gateway: 192.168.0.1 (use your existing gateway)
  7. DHCP: DEactivate
  8. DNS server: 192.168.0.1 (use your existing DNS server)
  9. activate changes: yes
  10. The fresh server will then restart with the above settings.

To update your server, login as user root on local console. Use 'top' command to display running jobs. Wait until 'yum' stopped running.

yum clean all
yum update

The fresh server will be updated.

signal-event post-upgrade
signal-event reboot

The server will reboot. Afterwards it is ready for installation of Affa 3.

Login as root on local or remote SSH console of affabox.



For SME10

There is a new version 3.3.1

However. Affa is a very complicated piece of work with a huge number of options and many have probably not been sufficiently tested depite requests for testing and feedback. Most of the major components should work but there may well be a plethora of edge cases that fail.


Warning.png Warning:
Do not rely on this as your sole backup. Make sure you have another type of backup as well



Warning.png Warning:
Do not use ANY Koozali SME v9 affa versions with v10. It will probably break both your backups and your new server


We have done very minimal testing on migration from v9 to v10.

See https://bugs.contribs.org/show_bug.cgi?id=11024


Under normal circumstances you would execute this to install :

yum install smeserver-extrarepositories-epel smeserver-extrarepositories-openfusion
signal-event yum-modify
config set UnsavedChanges no
yum install --enablerepo=smecontribs,epel,openfusion smeserver-affa

During development install can be executed with:

yum install --enablerepo=smedev,epel,openfusion smeserver-affa


Please post bug reports in the bug tracker

The server again needs to be updated.

signal-event post-upgrade
signal-event reboot

The server will reboot. Afterwards Affa 3 is ready for use.


Warning.png Warning:
Note that on SME10, a SSH AutoBlock feature interferes with the Affa requirement for regular SSH logins! You might have to disable this feature, aor increase the MaxAuthtries to allow Affa to operate correctly.


Creation of backup jobs

Job configurations of Affa 3 are no longer stored in an e-smith style database. They are stored in configuration files located in /etc/affa . Create your configuration files in this directory, e.g.

touch /etc/affa/backup-smeserver.conf

where backup-smeserver is your jobname, and add content as described below.

Quick start example

You have a SME production server with hostname 'smeserver‘ and IP 192.168.0.2.
You have a second SME box as your backup server with hostname 'affabox‘ and IP 192.168.0.10.

Login to your 'affabox' as root and edit /etc/affa/backup-smeserver.conf . Using e.g. editor nano, create the following example job configuration file for jobname backup-smeserver :

[backup-smeserver]
remoteHostName=192.168.0.2
SMEServer=yes
Watchdog=yes
RPMCheck=yes
ConnectionCheckTimeout=120
Debug=no
Description=Backup of 192.168.0.2 smeserver.athome.xx
DiskSpaceWarn=strict
RootDir=/var/affa
TimeSchedule=0630
localNice=15
remoteNice=15
rsync--inplace=yes
rsyncCompress=no
rsyncTimeout=900
scheduledKeep=1
dailyKeep=7
weeklyKeep=4
monthlyKeep=12
yearlyKeep=1
status=enabled

If you have a Letsencrypt certificate on the production server, then you should also include

Include=/etc/dehydrated

Then save your job configuration file.

Now check that your configuration is OK:

affa --configcheck

This should throw out no errors. Now create / send key files to your productive smeserver:

affa --send-key backup-smeserver

The following output should appear on the console:

Job sme-backup: Generating RSA keys...
Successfully created RSA key pair.
root@192.168.0.2's password: <Enter password of 192.168.0.2/smeserver and hit ENTER>
Public key sent to 192.168.0.2

Now run your job manually, both for test purposes, as well as to create RSA keys:

affa --run backup-smeserver

The following output should appear on the console:

The authenticity of host 'backup-smeserver (192.168.0.2)' can't be established.
RSA key fingerprint is 3b..........65.
Are you sure you want ton continue connecting (yes/no)? <yes>

Now your backup job should run for the first time. Depending on the volume of the files, this takes quite a while.

Once the job is done, check that the archive is available:

affa –-list-archives

or run the job a second time:

affa --run backup-smeserver

Note that you won't be asked for the password again. Note as well, that this second run of the job backup-smeserver should run considerably faster, because not all files are copied again: just the difference to the previous run is backed up, the rest is stored as hard links to the existing file copies.

Check that the second archive is available too:

affa --list-archives

From here you are able to work as with Affa 2. Modify your config file as required and described below. Automate the backup using the command affa --make-cronjobs, see below.

Job configuration properties

Note 1: The default values shown in this table are the Affa program defaults and not to be confused with the preset values in the job configuration files, e.g. backup-smeserver.conf.

Note 2: The complete documentation of the settings of Affa3 is available in [1] . It should be taken in consideration!

Note 3: Affa 3 for SME is a fork of Affa 3 for CentOS. It adds a few options back again which have been removed. Generally the configuration properties as described here do apply. Arnaud added the following functions for SME:

  • The parameter “SMEServer”
  • The parameter and function “Watchdog”
  • The parameter and function “RPMCheck”
  • The functions “--rise” and “--undo-rise”

Consequently the list below should reproduce the list for the unforked Affa 3 version, plus adding the properties above. For details refer to Arnaud's website.

Property Value Multivalue Default Description
remoteHostName FQHN or IP no <none> FQHN or IP of the source host (mandatory)
remoteUser account no root The user account to be used with all ssh logins. May be case sensitive, e.g. Administrator account on Windows
Description text string no <none> Any text that describes the job
TimeSchedule HHMM yes 2230 The time the job will run. Use exactly 4 digits, no colon, no point. Important: Using the proper format HHMM is essential. Badly formatted TimeSchedule will cause strange Perl errors. Multiple times can be achieved by repeating the line, each line with a different time.
status enabled or disabled no enabled When set to disabled, no cron entries will made. You can still run a job manually.
Include full path yes <none> File(s) or directory(s) to be included in the backup.
Exclude full path yes <none> File(s) or directory(s) to be excluded from the backup.
RootDir full path no /var/affa where to store the backup archives. Do not use /home/e-smith or /root as these are included in the backup and therefore the rise option will not work! Recommended: /var/affa
scheduledKeep integer >= 1 no 1 how many of the scheduled backups to be kept
dailyKeep integer >= 0 no 7 how many of the daily backups to be kept
weeklyKeep integer >= 0 no 4 how many of the weekly backups to be kept
monthlyKeep integer >= 0 no 12 how many of the monthly backups to be kept
yearlyKeep integer >= 0 no 2 how many of the yearly backups to be kept
EmailAddress name@domain.com yes root comma separated list of mail addresses, where the messages should be sent to
Note: By default Affa only sends messages on errors, never on success (see property chattyOnSuccess).
RetryAttempts integer >= 0 no 4 When set to a value>0, Affa re-run a failed job RetryAttempts times with a delay of RetryAfter seconds.
RetryAfter seconds >= 0 no 900 when set to a value>0, wait RetryAfter seconds before re-running the job after an error. Only applicable with RetryAttempts>0
RetryNotification yes or no no no when set to no, Affa does not send an error message when a job has failed and RetryAttempts is configured. An error message is only sent when the last attempt has failed.
NRPEtrigger hours no 24 NRPE reports a job as critical when the last successful run is older then NRPEtrigger hours. To exclude the job from monitoring set to a value < 0.
SambaShare yes or no no no Access to the job archives via CIFS protocol.
SambaValidUser local account yes affa User who has permission to access the job archives via the CIFS protocol.
preJobCommand
preJobCommandRemote
path relative to /etc/affa/scripts/ yes <none> Scripts to be executed before a job run. The job name and type (scheduled, daily etc.) are passed as arguments to the program. The preJobCommandRemote scripts are copied to the remote server and then executed there. The scripts are executed in alphabetical order. Use a numerical prefix if you need a specific order e.g. 01scriptB, 02remotescriptA, 03scriptA. Use the included prescript-sample.pl and prescriptRemote-sample.pl as a starting point for own scripts.
Note: If you use the parameter “SMEServer=yes”, you may delete the “Included” relative to SME default directories in the jobconfig.ini: they will be backuped automatically.
postJobCommand
postJobCommandRemote
path relative to /etc/affa/scripts/ yes <none> Scripts to be executed after a job run. The job name, the type (scheduled, daily etc.) and the exit code of the job run are passed as arguments to the program. The postJobCommandRemote scripts are copied to the remote server and then executed there. The scripts are executed in alphabetical order. Use a numerical prefix if you need a specific order e.g. 01scriptB, 02remotescriptA, 03scriptA. Use the included postscript-sample.pl and postscriptRemote-sample.pl as a starting point for own scripts.
Note: If you use the parameter “SMEServer=yes”, you may delete the “Included” relative to SME default directories in the jobconfig.ini: they will be backuped automatically.
dedup yes or no no no The purpose of the deduplication is to remove duplicate files to save backup space. When set to 'yes' file deduplication is run after the synchronization has been completed. It looks for files that have identical content, user, group and permissions and replace duplicates by hardlinks. Deduplication scans the just completed archive and the previous one, that usually is

scheduled.0 and daily.0 or scheduled.0 and scheduled.1. Consider this scenario: A user has renamed directories or files. Rsync sees those as new ones and copies them. Deduplication finds the identical copies in the previous archive and replace them by hardlinks. To use deduplication the Freedup program needs to be installed. Affa actually runs freedup -upg scheduled.0 <previous_archive>.

dedupKill yes or no no no When set to 'no' the job ignores affa --kill or affa --killall when deduplication is running. This is useful in the context of killAt which is typically used to stop bandwith utilisation.
DiskSpaceWarn strict or normal or risky or none no strict Checks disk space remainin on the backup device and issue a warning via email.
sshPort service port no 22 When sshd on the remote host listens on a non-standard port, set the port here.
ConnectionCheckTimeout seconds no 120 Affa checks the ssh connection before the rsync process is started and exits with an error after the configured time if the host did not respond.
BandwidthLimit integer>=0 kilobytes per second no 0 Limits the data transfer rate. A value of zero specifies no limit.
rsyncTimeout seconds no 900 Rsync exits after the configured time if no data was transferred. This avoids infinitely hanging in case of a network error.
rsyncCompress yes or no no yes Compress the transferred data. May be useful with slow internet connections. Increases CPU load on remote and backup host.
rsync--inplace yes or no no yes Set to no if the rsync version on the remote hist does not support this option.
rsync--modify-window integer >= 0 no 0 When comparing two timestamps, rsync treats the timestamps as being equal if they differ by no more than the modify-window value. This is normally 0 for an exact match. A value >= 0 is useful if you can't get the clocks of the remote host and the Affa server in sync.
rsyncOptions string no <none> Additional option string to be passed to rsync
localNice -19...+19 no 0 run rsync local process niced.
remoteNice -19...+19 no 0 run rsync process on source niced.
killAt HHMM no <none> The time at which a job will be killed if it was still running. You can use it for example to kill jobs that are running over the internet early in the morning so that your users have the full bandwidth available during office hours.
resumeKilledAt HHMM no <none> The time at which a killed job will be resumed. This allows you to start earlier in the evening than the scheduled time.
chattyOnSuccess integer >= 0 no 0 When set to a value>0, Affa sends a message on a successfully completed job run and decrements the chattyOnSuccess value. When the value has reached zero, Affa falls back to the default and only sends messages on errors.
AutomountDevice
AutomountPoint
full path no <none> Device and mountpoint of backup device (e.g. USB disk). Device is automounted before a job starts and unmounted after job completion. With both properties empty no automount is done.
AutomountOptions string no <none> An option string passed to the mount command.
AutoUnmount yes or no no yes When set to 'no' the automounted device stay mounted after the Affa run.
Debug yes or no no no Set to yes to increase log verbosity.
remoteRsyncBinary full path no /usr/bin/rsync If the rsync program on the remote server is located in non-standard location, set it here
remoteNiceBinary full path no /bin/nice If the nice program on the remote server is located in non-standard location, set it here.
localRsyncBinary full path no /usr/bin/rsync If the local rsync program is located in non-standard location, set it here
localNiceBinary full path no /bin/nice If the nice local program is located in non-standard location, set it here.
RemoteAuthorizedKeysFile path relative to remote user's home or full path no .ssh/authorized_keys2 If the remote host stores the authorized keys file in a non-standard location, set it here.
rsyncdMode yes or no no no Set to yes to connect to the rsync daemon on the remote host (instead of running rsync over ssh).
rsyncdModule string no AFFA The rsyncd module name (only applicable with rsyncdMode=yes).
rsyncdUser string no affa The username for authentication to the rsync daemon (only applicable with rsyncdMode=yes).
rsyncdPassword string no <none> The password for authentication to the rsync daemon (only applicable with rsyncdMode=yes).
globalStatus enabled or disabled or jobs no jobs Set to enabled or disabled to overide the status settings in all job sections. When set to value jobs, the status settings in the job sections are effictive.
Note: This property is allowed only in the [GlobalAffaConfig] section.
sendStatus daily or weekly or monthly or never no weekly Defines how frequently the status will be sent.
Note: This property is allowed only in the [GlobalAffaConfig] section.
SMEServer yes or no no no When set to yes, the default e-smith directories are automatically included and the property RPMCheck=yes can be used.
Note: this property is not implemented in Affa 3 for CentOS. It has been specifically added to the fork for SME.
RPMCheck yes or no no no Only applicable to jobs that backup a SME server. Compares the packages installation of the source host with this affa backup host. Sends a message with diff list if not in sync. This check is useful, if you want have the option to rise the backup server to a production server from a backup.
Note: this property is not implemented in Affa 3 for CentOS. It has been specifically added to the fork for SME.
Watchdog yes or no no yes Only applicable to jobs that backups a SME server. When a job is started, affa installs a watchdog script on the source in /etc/cron.d/, which sends a warning message, if the next scheduled job (taken from the TimeSchedule property + 10 minutes) did not run. This guarantees, that you will be notfied even in case of a affa server outage. The watchdog script send a daily reminder message, if the error continues. The next run job replaces the watchdog script with a new trigger time.
Note: this property is not implemented in Affa 3 for CentOS. It has been specifically added to the fork for SME.

Default configuration properties

For all 'name=value' properties defaults can be set in the [GlobalAffaConfig] section which are used in all job configuration when the corresponding property is omitted. For example, when these properties are set in [GlobalAffaConfig] section, they can be omitted in the specific job configurations. This is useful, when you set up many similar jobs.

Example: You want to set the property 'localNice' to 19 for all jobs. Then add the following section in one of your configuration files:

[GlobalAffaConfig]
localNice=19

Consequently you don't need to set this property for your individual jobs. Properties set in the job record override the defaults.

Properties for global Settings

The following special properties are only applicable to the [GlobalAffaConfig] section:

  • sendStatus
  • globalStatus

All jobs can be disabled for execution through 'cronjobs" with setting 'globalStatus' to 'disabled'.

[GlobalAffaConfig]
globalStatus=disabled

To re-enable run, either delete the above configuration line, or set to:

globalStatus=jobs

which is the default value.

You can also set the value to

globalStatus=enabled

which enforces the the job execution through 'cronjobs' and overrides the specified values in the section for the specific job.

Usage and command line options

Important.png Note:
Options can be abbreviated to uniqueness, e.g. --mak is equal to --make-cronjobs


affa --run JOB

Starts a job run. Usually done by the cronjob.

affa --make-cronjobs

Configures the cronjobs as scheduled in the jobs records. Run this command to make changes of time related properties effective i.e. TimeSchedule, killAt and resumeKilledAt properties. By default this command will by executed by 'cronjobs' every 15 minutes. So you don't need to run this command manually, you can also just wait max. 15 minutes before your updates job configurations become effective.

affa --configcheck

Checks the syntax and values in all configuration files found in /etc/affa/. Run this command after modifying the configuration. Lower/Upper case errors in property names are corrected automatically.

affa --send-key [JOB JOB ...]

This first generates the RSA key on the Affa Server, if not already done. Then the public key is send to the hosts 'remoteHostName' as defined in section of each job JOB and generates the job specific ssh known host entry.

Important.png Note:
When initially doing this step, you will need to temporarily enable "Allow secure shell access using standard passwords" on the source server.


affa --check-connections [JOB JOB ...]

Checks the ssh login and if applicable rsyncd auth for all jobs given as arguments. Without any arguments all jobs are checked.

affa --full-restore [--preserve-newer=no] [--delete=yes] JOB [ARCHIVE]

Does a full restore remote server of all backed up files and directories from the backup ARCHIVE. If ARCHIVE is not given, the archive 'scheduled.0' is used as the default. The full restore reconstructs the server as it was at the time of the backup. After the restore the source host reboots.

With option --preserve-newer=yes files on the remote server with modification time newer than on the backup are not overwritten.

With option --delete=yes all files on the remote server, which are not in the backup, are deleted.

If the parameter “SMEServer=yes” is set, the signal-event pre-restore and signal-event post-upgrade will be used automatically.

affa --list-archives JOB

Displays a table of all present archives of job JOB with date, number of files, size and and bytes received. While column buTime shows the actual rsync time, the column ddTime shows the length of the subsequent deduplication run. ddYld is the gained deduplication yield in bytes.

Affa version 3.1.0-0 on affa-2.mydomain.com
+------------------------------------------------------------------------------+
| Job: fshare-ak                                                               |
| Description: Fileserver AK                                                   |
| Directory: /var/affa/fshare-ak/                                              |
| Hostname: 10.204.104.4                                                       |
| Email: sysadmin@mydomain.com                                                 |
+-----+----------------------+--------+--------+-------+-------+-------+-------+
| Run | Completion date      | buTime | ddTime | ddYld | Files | Size  | Recvd |
+-----+----------------------+--------+--------+-------+-------+-------+-------+
| Y 0 | Wed 2010-06-23 20:26 | 11m53s |      - |     - |  412k |  143G |  470M |
+-----+----------------------+--------+--------+-------+-------+-------+-------+
| M11 | Sun 2010-08-29 20:22 |  7m50s |      - |     - |  417k |  153G |   14M |
| M10 | Sun 2010-09-26 20:23 | 8m57s  |      - |     - |  430k |  156G |   14M |
| M 9 | Sun 2010-10-31 20:25 | 10m05s |      - |     - |  448k |  161G |   15M |
| M 8 | Sun 2010-11-28 20:31 | 16m42s |      - |     - |  463k |  167G |   17M |
| M 7 | Sun 2011-01-02 20:48 | 33m07s |      - |     - |  486k |  173G |  1.1G |
| M 6 | Sun 2011-01-30 20:33 | 18m54s |      - |     - |  493k |  176G |   24M |
| M 5 | Sun 2011-02-27 20:28 | 13m31s |      - |     - |  490k |  176G |   19M |
| M 4 | Sun 2011-03-27 20:28 | 13m08s |      - |     - |  491k |  175G |   17M |
| M 3 | Sun 2011-05-01 20:30 | 15m41s |      - |     - |  493k |  179G |   18M |
| M 2 | Sun 2011-05-29 20:28 | 13m40s |      - |     - |  494k |  182G |   31M |
| M 1 | Sun 2011-06-19 20:26 | 11m20s |      - |     - |  493k |  183G |   17M |
| M 0 | Sun 2011-07-03 20:28 | 13m46s |      - |     - |  496k |  183G |   18M |
+-----+----------------------+--------+--------+-------+-------+-------+-------+
| W 3 | Sun 2011-07-10 20:28 | 13m38s |      - |     - |  479k |  180G |   16M |
| W 2 | Mon 2011-07-11 20:35 | 20m27s |      - |     - |  479k |  180G |  301M |
| W 1 | Fri 2011-07-15 20:30 | 15m53s |      - |     - |  480k |  180G |   62M |
| W 0 | Sun 2011-07-24 19:39 |  9m25s |      - |     - |  482k |  181G |   16M |
+-----+----------------------+--------+--------+-------+-------+-------+-------+
| D 6 | Thu 2011-07-28 19:42 | 12m22s |      - |     - |  483k |  182G |  176M |
| D 5 | Fri 2011-07-29 23:29 | 11m10s |  5h33m |   45G |  483k |  182G |   16M |
| D 4 | Sat 2011-07-30 19:53 | 23m26s |  2h30m |  8.3G |  483k |  182G |   17M |
| D 3 | Sun 2011-07-31 20:07 | 37m31s |  4m47s |  8.3G |  483k |  182G |   17M |
| D 2 | Mon 2011-08-01 20:44 |  1h14m |  7h50m |  8.5G |  484k |  182G |  630M |
| D 1 | Tue 2011-08-02 20:02 | 32m28s | 12h20m |  8.3G |  484k |  182G |   74M |
| D 0 | Wed 2011-08-03 19:58 | 28m46s | 11h01m |  8.5G |  484k |  182G |  214M |
+-----+----------------------+--------+--------+-------+-------+-------+-------+
| S 0 | Thu 2011-08-04 20:00 | 30m28s | 11h52m |  8.5G |  484k |  182G |  203M |
+-----+----------------------+--------+--------+-------+-------+-------+-------+
affa --list-archives [--csv] JOB

With --csv, the output is in machine readable colon separated format.

Archive:Count;Date;Files;Size;RootDirFilesystemAvail;RootDirFilesystemUsed;valid;TotalBytesReceived;ExecutionTime;DedupTotalFiles;DedupReplacedFiles;DedupSavedBytes;DedupExectime;DedupDate
monthly;00000;201509270631;393237;37252509103;166889260;47876936;yes;1327370;1160638358;99;;;;;
weekly;00003;201510040531;390273;37042612135;164566692;50199504;yes;1117726;45430532;59;;;;;
weekly;00002;201510110531;395553;37299589800;162612204;52153992;yes;1117092;92369237;64;;;;;
weekly;00001;201510180531;403831;37707599172;161855844;52910352;yes;1317083;148317764;64;;;;;
weekly;00000;201510250531;406509;38146200127;161041156;53725040;yes;1104759;64083948;63;;;;;
daily;00006;201510310531;415663;39248862982;162979176;51787020;yes;1958110;198060283;66;;;;;
daily;00005;201511010531;411014;38563389171;163061136;51705060;yes;1156506;48903675;75;;;;;
daily;00004;201511020531;408910;38612857040;162359124;52407072;yes;1007434;210812387;75;;;;;
daily;00003;201511030531;410615;37815861577;162026060;52740136;yes;1524069;162697515;67;;;;;
daily;00002;201511040531;414784;38131734213;161990024;52776172;yes;1571657;134250735;63;;;;;
daily;00001;201511050531;420780;38433895988;161906580;52859616;yes;2021507;149388808;103;;;;;
daily;00000;201511060531;425920;38677614350;161778812;52987384;yes;1987971;166510621;71;;;;;
scheduled;00000;201511070531;425663;38815712018;161612600;53153596;yes;2003540;139603231;68;;;;;
affa --status [--csv]

Displays a table of all configured jobs with enable status, time of last and next run, size of the most recent archive, exectution time of the last run and the number of scheduled (S), daily (D), weekly (W), monthly (M) and yearly (Y) archives. If last time shows 'failed', the job did not run in the last 24h. For disabled jobs 'Last' always shows 'failed' after 24 h. To see the date and time of the last run of those jobs use the --list-archives option. Column 'Next' shows the time when the next run will be started, if 'Enabled' is 'yes'. Column 'ddYld' shows the deduplication yield in bytes.

Affa version 3.1.0-0 on affa-2.mydomain.com
+--------------+-----+-------+--------+-------+-------+-------+----------------+
| Job          | ENA |  Last |   Time |  Next |  Size | ddYld | N of S,D,W,M,Y |
+--------------+-----+-------+--------+-------+-------+-------+----------------+
| ads-ak       | yes | 20:20 |  0m29s | 20:20 |   28M |     - |  1, 7, 2, 0, 0 |
| ak-user-ma.. | yes | 22:31 |  1m51s | 22:30 |  170M |     - |  3, 7, 4, 6, 0 |
| azubi-1      | yes | 03:39 |  3h24m | 00:15 |   44G |  1.2G |  1, 7, 4,10, 1 |
| dcpant       | yes | 03:06 |  1m03s | 03:05 |  1.3G |     - |  1, 7, 4,10, 1 |
| eshare       | yes | 21:06 | 21m01s | 20:45 |   24G |  178M |  1, 7, 4,10, 1 |
| etherpad     | yes | 20:40 |  0m13s | 20:40 |  3.1M |     - |  1, 7, 4, 2, 0 |
| fazubi       | yes | 08:16 | 11h16m | 21:00 |  132G |  3.3G |  1, 7, 4,10, 1 |
| fschare-rh   | yes | de-duplicating (pid 9719)              |  1, 7, 4,10, 1 |
| fsh-02       | yes | 07:53 | 12h23m | 19:30 |  182G |  8.5G |  1, 7, 4,10, 1 |
| fshare-ak2   | yes | 00:30 |  0m26s | 00:30 |  415M |  544k |  1, 7, 4,10, 1 |
| helpdesk     | yes | 21:27 |  2m16s | 21:25 |  895M |  138k |  1, 7, 4,10, 0 |
| it-share     | yes | running rsync (pid 9744)               |  1, 7, 1, 0, 0 |
| lightroom    | yes | waiting (pid 9528)                     |  1, 7, 4, 2, 0 |
| localhost    | yes | 02:15 |  0m08s | 02:15 |  395k |     - |  1, 7, 2, 0, 0 |
| mediawiki    | yes | 16:10 |  0m06s | 19:40 |  1.1G |  6.7M |  3, 7, 4,10, 1 |
| mshare       | yes | 00:48 |  1h33m | 23:15 |   18G |  2.4G |  1, 7, 4,10, 1 |
| wshare       | yes | 00:34 |  1h49m | 22:45 |   23G |  484M |  1, 7, 4,10, 1 |
+--------------+-----+-------+--------+-------+-------+-------+----------------+
2 disabled jobs not listed. Use --all to display.

With --csv, the output is in machine readable colon separated format.

affa --show-config-pathes [--csv] [JOB JOB ...]

Prints the full pathes of the file where the section of job JOB is defined.

affa --show-default-config

Prints a list of all allowed property names with their default values. These values are used, when omitted in the job sections.

affa --show-schedule [--all]

Prints a 'graphical' timetable for all enabled jobs. The resolution is 30 minutes. An 'S' character marks the scheduled start times. The duration of the job runs are marked with '=' characters.

Affa version 3.1.0-0 on affa-2.mydomain.com
              TIME 12:00    16:00    20:00    0:00     4:00     8:00
            fsh-02 -------- -------S ~~~~~~~~ ~~~~~~~~ ~~~~~~~~ --------
            ads-ak -------- -------- S------- -------- -------- --------
          etherpad -------- -------- -S------ -------- -------- --------
            eshare -------- -------- -S~----- -------- -------- --------
            fazubi -------- -------- --S~~~~~ ~~~~~~~~ ~~~~~~~~ ~-------
          helpdesk -------- -------- --S----- -------- -------- --------
          it-share -------- -------- ---S=~~~ ~~~~~~~~ ~~~~~~~- -------- busy
            wshare -------- -------- -----S~~ ~~------ -------- --------
ak-user-management -------- -------- -----S-- -------- -------- --------
        fschare-rh -------- -------- -----S=- -------- --~----- -------- busy
            mshare -------- -------- ------S= ~~------ -------- --------
           azubi-1 -------- -------- -------- S=~~~~~~ -------- --------
        fshare-ak2 -------- -------- -------- -S------ -------- --------
         mediawiki -------- -------- -------- ---S---- -------- ----S---
         localhost -------- -------- -------- ----S--- -------- --------
            dcpant -------- -------- -------- ------S- -------- --------
         lightroom -------- -------- -------- -------- -------- S------- busy
Symbols: S=scheduled K=kill R=resume '='=rsync '~'=dedup
2 disabled jobs not listed. Use --all to display.
affa --log-tail [JOB]

Displays the tail of the logfile of job JOB with live update. This command is identical to tail -n 50 -f /path/to/logfile. Without the JOB argument the global logfile is shown.

affa --send-status

Sends the status table, the disk-usage and the archive list of all jobs to the email addresses configured in the [GlobalAffaConfig] section.

affa --disk-usage

Shows the current disk usage of all root dir filesystems

Affa version 3.1.0-0 on affa-2.mydomain.de
+------+--------+--------+----------------------------------------------------+
| Use% |   Used |  Avail | Root Dir                                           |
+------+--------+--------+----------------------------------------------------+
|  71% |  938GB |  365GB | /var/affa                                          |
|  20% |  194GB |  759GB | /mnt/affadev                                       |
+------+--------+--------+----------------------------------------------------+

With --csv, the output is printed in a machine readable colon separated format.

affa --cleanup JOB

After you have lowered a keep value, e.g. scheduledKeep, then archives with a higher indices will no longer be shifted and will exist for ever. This option finds these archives and deletes them after confirmation. When running the cleanup command on a locally attached USB drive, manually mount the USB drive BEFORE running this command. When finished, manually unmount the USB drive. This requirement may apply to some of the other commands listed here.

affa --rename-job JOB NEWNAME

Renames the job JOB to NEWNAME including the section name and archive directories.

affa --move-archive JOB NEWROOTDIR

Moves the archive directory of job JOB to the rootdir NEWROOTDIR and adjusts the value of property RootDir. NEWROOTDIR must be a full path starting with a slash. As moving across filesystems (e.g. from an external USB drive to the local disk) is not possible, Affa uses a copy command in this case and deletes the source directory after that. Depending on the archive size, copying across filesystems can take a long time.

affa --delete-job [--revoke-key] JOB

Irreversibly deletes all archives, of job JOB and set the status property to disable. With --revoke-key option, the public key on the remote server will be deleted.

affa --revoke-key JOB

Deletes the public RSA key on the remote server.

affa --kill JOB

Terminates the running job JOB and all its child processes (rsync processes).

affa --killall

Terminates all running jobs.

affa --mailtest JOB

Sends a test email to the email addresses configured in the JOB section. Use this to verify, that your mail system is working.

Important.png Note:
By default Affa only sends messages on errors, never on success (see property chattyOnSuccess).


affa --nrpe [JOB JOB ...]

Checks for jobs that were not run the last NRPRtrigger hours and reports them as failed. A Nagios/ICINGA NRPE complient message is printed and exit status returned.

affa --version

Displays the Affa version number and checks Samba and Freedup installation.

affa --warranty

Displays the disclaimer of liability.

affa --license

Displays the license Affa is released under.

affa --help

Displays a short help.

affa --debug

Enables verbose logging. Overrides job and global configurations.

SME Specific usage and command line options

These commands are implemented in a fork of Affa 3 project for use with SME. They are not part of the Affa 3 for CentOS project.

SME server 'rise' function

Important.png Note:
Before trying a rise please check /var/affa/YourServer/rpms-missing.txt

You may find there are missing rpms/contribs that template fragments depends on.

It may be necessary to install some of these rpms before a rise to prevent issues.


The SME server version of Affa has a unique feature restore feature enabling you to turn your Affa backup server into a replica of your server. It is an extremely fast way to restore your server.

affa --rise [--all] JOB [ARCHIVE]

Runs a full restore on the Affa server (!!!) of all standard files and directories from the backup ARCHIVE of job JOB. In other words: After completion, the Affa box reboots as a clone of the source server. Ensure, that the source server has been powered off before you reboot the Affa box, otherwise the network interface will not come up. This is important, when you run --rise remotely. The --rise feature only works with SME servers and should only be used on dedicated backup servers.

With option --all, all files and directories of the archive as defined by the include[] properties are restored. Files or directories with the same name on the Affa server will be overwritten and cannot be restored by a --undo-rise. This should not be an issue on a dedicated Affa server which does not hold any other data. After a possible --undo-rise those additional restored data must be removed manually.

Please note, that the rise process backs up the the Affa server itself before doing the restore from the archive. This backup is used by a possible --undo-rise run to restore the Affa server. Only the standard files and directories are backed up. Data in non-standard locations (like /opt) are untouched and will still exist after the rise run if they don't get overwritten by data of the backup (=if the backup contains /opt too e.g.).


Important.png Note:
The above command is not implemented in Affa 3 for CentOS. It has been specifically added to the fork for SME.


affa --undo-rise

This feature reverts a risen Affa box to a backup server. After a reboot, all configured jobs based on standard files and directories will work again.


Important.png Note:
The above command is not implemented in Affa 3 for CentOS. It has been specifically added to the fork for SME.


Example setups

Dedicated backup server

  • Setup a dedicated server and install Affa 3 for SME.
  • Setup a job for every server you want to backup.
  • Send the public keys to every server.
affa --send-key JOBNAME
  • Check whether password-less logins are working.
affa --check-connections
  • Check whether the scheduled jobs are evenly distributed over the day.
affa --show-schedule
  • Create the cron jobs.
affa --make-cronjobs
  • Check the status after 24 hours.
affa --status

Backup of single ibays

Suppose you want to backup the ibays 'staff1' and 'staff2' on your production server with WAN IP 82.123.1.1 to an Affa server in a different location over the internet every night at 2:30am.

  • Log into the Affa server.
  • Create a dedicated job file /etc/affa/ibay-staff.conf . Edit it and set
[ibay-staff]
Description=Backup of ibay-staff on 82.123.1.1
remoteHostName=82.123.1.1
TimeSchedule=0230
SMEServer=no
Include=/home/e-smith/files/ibays/staff1
Include=/home/e-smith/files/ibays/staff2
  • Save the configuration
  • send the public key to the production server
affa --send-key ibay-staff
  • check next morning
affa --list-archives ibay-staff
affa --status
ls /var/affa/ibay-staff

Two production servers backup each other

You have two sites connnected via a VPN and a SME Server running on each site. In that case you don't need a dedicated Affa backup server. Both production servers can additionally act as Affa backup servers backing up the server of the other site. Simply install Affa and configure a job that backs up the other one. You can use all Affa features except of the rise feature.

When using the rise feature, the server becomes any of the backed up systems, which is less useful in this scenario as it would give you a running copy of the server of the other site while the server of this site is down.

To get redundancy and a faster restore you can configure a local backup to an external USB or NAS device.

Please pay attention, that you do not backup the archives back to the other site. Set the RootDir property to a path which is not included in the SME default backup list. When leaving the RootDir property to its default '/var/affa', this is guaranteed.

Use Affa to backup to a NFS-mounted NAS or a local attached USB drive

Important.png Note:
This chapter still needs to be checked whether it works with Affa 3 for SME.


You want to backup your SME production server with hostname 'prodbox‘ and IP 10.200.48.1 on a mounted filesystem instead of setting up a dedicated Affa box.

Setup NAS

You have a FreeNAS box with IP 10.200.48.2 up and running with NFS service enabled. The disk is mounted to /mnt/affashare. You have authorized the IP address of your prodbox server to access share /mnt/affashare.

  • log into the 'prodbox' and install the NFS packages
/usr/bin/yum install --enablerepo=smecontribs smeserver-nfs
  • now enable and start the portmapper service
config setprop portmap status enabled
service portmap start
  • mount the NFS share
mkdir -p /mnt/affadevice
mount 10.200.48.2:/mnt/affashare /mnt/affadevice


Alternatively setup a USB drive
  • log into the 'prodbox'
  • connect a USB hard disk to the USB Bus. Now you must determine what device the kernel has assigned to the drive. View the /var/log/message and search for Initializing USB Mass Storage driver. A few lines below you'll find the name of the device. In this example it is sdh. Replace /dev/sdh by your device in following instructions.
  • use the fdisk program to create a linux partition. Verify that this is really the attached USB drive before you continue!
fdisk /dev/sdh

You'll most likely find an existing vfat dos partition, which you have to delete first. In the following we assume, that you have created a single partition /dev/sdh1.

  • now format the drive with an ext3 filesystem
mkfs.ext3 /dev/sdh1
  • make the mount point
mkdir -p /mnt/affadevice
  • add the following line to the /etc/fstab
/dev/sdh1 /mnt/affadevice ext3 defaults
  • mount the drive
mount /mnt/affadevice
  • crosscheck your work using the df command
df
Setup Affa

You want to run backups at 11:30 h, 15:30 h and 19:30 h and you want to keep the last 3 scheduled backups, 7 daily, 5 weekly, 12 monthly and 1 yearly backups.

  • log into the 'prodbox' and install the Affa packages as described above.
  • create/edit the conf file of the job: nano /etc/affa/prodbox.conf

and set:

[prodbox]
remoteHostName=localhost
TimeSchedule=1130
TimeSchedule=1530
TimeSchedule=1930
scheduledKeep=3
dailyKeep=7
weeklyKeep=5
monthlyKeep=12
yearlyKeep=1
RootDir=/mnt/affadevice

Review the other properties and change them to your needs.

  • run the job manually
affa --run prodbox
Limitations

With this kind of setup you cannot use the affa rise feature, as it requires the backup archive to be located on the same fileystem as the server installation. The rise option uses hardlinks, which are not working across filesystems.

Automount

Having the backup archives in the same filesystem is always a risk, which can be minimized by using the automount feature. Then the external filesystem is only mounted during a job run.

In the NAS example set

AutomountDevice=10.200.48.2:/mnt/affashare
AutomountPoint=/mnt/affadevice

and skip the step 2.

In the USB drive example set

AutomountDevice=/dev/sdc1
AutomountPoint=/mnt/affadevice

and skip the steps 5 to 8.

The mount point will be automatically created, if it does not exist.
To access the archive directory, you need to mount it manually.

Copying a AFFA USB hard drive archive to a new disk

Affa uses copious amounts of hard links to compress and preserve disk space for its backups. If you are in the situation where you want to copy such a disk archive to a new (bigger) disk, you need to ensure that the hard links are copied correctly or the destination copy may became significantly bigger than the total of the source archive.

One way to copy across file systems (i.e. two different USB disks) and preserve the hard links is as follows:

  • mount both USB drives but with different mount points. e.g. /media/backup1 & /media/backup2 and then:
mkdir /media/backup2/archive 
cd /media/backup1/archive
tar cpf - . | ( cd /media/backup2/archive && tar xpf - )

where archive is the name of the AFFA job you want to move to the other disk.

Restore

Restore single files or directories

Example 1: It's Tuesday January 6th 2009, when user 'briedlin' asks you to restore the messages of his mailbox 'orders' he has accidentally deleted on Monday.

  • You first must check what backup archives are available. The jobname of this server backup is 'primmail'. To get a listing of all archives run
affa --list-archives primmail

(see the example listing in chapter Affa#Usage_and_command_line_options

  • Choose the daily.0 archive, which was created Monday night. Now restore the mailbox 'orders' using the rsync command.
  • Now run the rsync command (note the trailing slash!) on the Affa backup server:
export RDIR=/home/e-smith/files/users/briedlin/Maildir/.orders/  # this variable is used to shorten the next command line
rsync -av /var/affa/primmail/daily.0/$RDIR 10.204.48.1:$RDIR

If the servers are configured to use a different ssh port eg 2222, then instead do:

export RDIR=/home/e-smith/files/users/briedlin/Maildir/.orders/  # this variable is used to shorten the next command line
rsync -av -e 'ssh -i /root/.ssh/id_rsa_affa -p 2222' /var/affa/primmail/daily.0/$RDIR 10.204.48.1:$RDIR


Example 2: A user has deleted the file orderform.pdf from ibay 'docs' on the server 10.204.48.1 and asks you to restore it.

  • You have searched and found the latest version of this file in weekly archive of job 'prodserv'.
  • To copy it back to the server 10.204.48.1 run on the Affa server
export RFILE=/home/e-smith/files/ibays/docs/files/orderform.pdf  # this variable is used to shorten the next command line
rsync -av /var/affa/prodserv/weekly.1/$RFILE 10.204.48.1:$RFILE
  • If the servers are configured to use a different ssh port eg 2222, then instead do:
export RFILE=/home/e-smith/files/ibays/docs/files/orderform.pdf  # this variable is used to shorten the next command line
rsync -av -e 'ssh -i /root/.ssh/id_rsa_affa -p 2222' /var/affa/prodserv/weekly.1/$RFILE 10.204.48.1:$RFILE


Example 3: Restoring a file with special characters in the path

Do not use shell variables to shorten the command. It would complicate things more than it would help. Quote the source and destination path. In the destination path escape blank characters with a double backslash and brackets with a single backslash. On the Affa backup server do:

rsync -av "/var/affa/fileshare/daily.4/home/e-smith/files/ibays/mechfiles/files/Valve Control (Design Gr)/VALVE LIST FOR ISSUED.xls" "192.168.1.7:/home/e-smith/files/ibays/mechfiles/files/Valve\\ Control\\ \(Design\\ Gr\)/VALVE\\ LIST\\ FOR\\ ISSUED.xls"

If the servers are configured to use a different ssh port eg 2222, then instead do:

rsync -av -e 'ssh -i /root/.ssh/id_rsa_affa -p 2222' "/var/affa/fileshare/daily.4/home/e-smith/files/ibays/mechfiles/files/Valve Control (Design Gr)/VALVE LIST FOR ISSUED.xls" "192.168.1.7:/home/e-smith/files/ibays/mechfiles/files/Valve\\ Control\\ \(Design\\ Gr\)/VALVE\\ LIST\\ FOR\\ ISSUED.xls"

Full restore

Important.png Note:
Please for notes about rpms-missing.txt above: https://wiki.contribs.org/Affa#SME_Specific_usage_and_command_line_options


Generally:

affa --full-restore [optional settings] JOB [ARCHIVE]

This rsyncs the data (files and directories )from the backup ARCHIVE back to the 'remoteHostname' defined in the configuration of the job JOB. If ARCHIVE is not given, the archive 'scheduled.0' is used as the default. The --full-restore reconstructs the server as it was at the time of the backup and takes following IMPORTANT optional settings in consideration:

With option [--preserve-newer=yes] files on the remote server with modification time newer than on the backup are not overwritten.

With option [--delete=yes] all files on the remote server, which are not in the backup, are deleted.

If the parameter “SMEServer=yes” is set, the signal-event pre-restore and signal-event post-upgrade will be used automatically.

After the restore is done, the restored server reboots automatically.

Example: You have backuped your production server 'prodsrv' as job 'prodbox'. To restore only lost data from the latest backup run

affa --full-restore prodbox

To restore exactly from the older archive daily.3 run

affa --full-restore [--preserve-newer=no] [--delete=yes]  prodbox daily.3


Warning.png Warning:
A full restore with --preserve-newer no and --delete yes reconstructs the server as it was at the time of the backup. That means, that all files created or server configuration changes made after the backup will be lost!


Moving a SME server installation to new hardware using the Affa rise feature or the backup-restore functions

Please see this Howto: Moving SME to new Hardware

Restore from USB drive on new server

This tip comes from http://forums.contribs.org/index.php?topic=42412.0

Q) I have complete backups using affa stored on a usb hard drive connected to our affa backup server. I need to restore an earlier monthly backup of our job "mailbackup" to a test server rather than back to the original system. If I did it from the backup server I see the instructions of how to rise that server to the current backup on that server but I want to restore a point about a month ago before some strange things happened. And I want to do it on a machine that is not our backup server or our production server. I tried to figure out how but am lost in the options. My goal is to do some testing.

A) On your testserver setup a job "mailbackup" identical to that on your backup server (a copy of the conf file is available into the folder of the archive, as an hidden file .mailbackup-setup.ini) but set property RootDir to /var/affa and property status to disabled. Connect the USB drive and copy the archive of the job mailbackup to /var/affa. Then run affa --rise mailbackup ARCHIVE

FAQ

What files and directories are included by default?

With SMEServer=no nothing at all.

With SMEServer=yes the SME default backup list is the output of following command:

perl -e 'use esmith::Backup;$b=new esmith::Backup;print join("\n",$b->restore_list)."\n"' 

and as additional folder:

/etc/affa


Important.png Note:
The following example needs to be reworked for Affa 3.


Can I exclusively backup image files from a specific directory? Yes. Assuming you want to backup all gif and jpg files from directory /home/e-smith/files/ibays/pictures/files use this configuration

db affa setprop JOBNAME SMEServer no           # do not include SME Server default directories and files
db affa setprop JOBNAME Include[0] '/home/e-smith/files/ibays/pictures/files' # start searching here
db affa setprop JOBNAME Include[1] '*/'        # search the tree recursively
db affa setprop JOBNAME Include[2] '*.gif'     # copy files that match these patterns
db affa setprop JOBNAME Include[3] '*.jpg'  
db affa setprop JOBNAME Exclude[0] '*'         # exclude all others
db affa setprop JOBNAME rsyncOptions '-m'      # do not create empty folders


How can I move a backup archive from my production server in a remote branch office to the Affa server using an USB disk? I want to avoid to download the first full backup over the internet. Install Affa on your production server, connect and mount the USB disk. Setup a job to write the archive to the USB disk. After the job run, connect the USB disk to your Affa server and setup the job in the same way. Use the setup script from the archive directory. You only need to change the remoteHostName property. Now use the --move-archive option to move the archive to the local disk. If not already done, send the public key to your production server. You're done.


How do I backup two SME servers behind a firewall? First you need to configure port forwardings for the ssh service on your firewall. Use a non-standard port, e.g 2200->22, for your second server. Setup a job on your Affa server for each of your production servers. Set up the job property sshPort=2200 for second server.

Uninstall

This removes the installed Affa package, all configuration data and all backup archives.

Run the following commands for each job:

affa --revoke-key JOBNAME
affa --delete-job JOBNAME

Verify that all jobs have been deleted:

affa --status

Remove the Affa package:

yum remove smeserver-affa-3.2.2.1-0.noarch.rpm

Cleaning up:

rm -f /etc/cron.d/affa
rm -rf /etc/affa
rm -rf /var/affa /var/log/affa

Troubleshooting

Affa stops working after prodserver was rebuild.

Description:

  • Affa stopped working or is broken.
  • When affaserver tries to make an ssh connection to prodserver it fails.
  • In the /var/log/affa/jobname.log file you will this message: "SSH connection to prodserver ip failed. Did you send the public key".
  • Resending the keys does not solve the problem.

Solution:

  • This is probably due to the key for prodserver that is stored on the affaserver no longer matches the key of prodserver.
  • You need remove the existing key so that the next ssh connection re-gets the keys from prodserver.
  • From the command line edit the /root/.ssh/known_hosts file and remove the line for prodserver. It starts with: prodserver ip ssh-rsa

Bug report

Affa is listed in the bugtracker contribs section. Please report all bugs, new feature requests and documentation issues there.


IDProductVersionStatusSummary (8 tasks)
12165SME Contribs10.0CONFIRMEDrsync update rsync.x86_64 0:3.1.2-11.el7_9 breaks AFFA sync
12084SME Contribs10.0RESOLVEDAFFA --run job ends with message "uninitialized value"
12081SME Contribs10.0RESOLVEDif host key verification fails (because response no), affa tries again 3
11784SME Contribs10.0VERIFIEDConf files placed in /etc/affa cannot contain spaces and/or sub folders with spaces
10467SME Contribs10.0CONFIRMEDNFR : before a rise check if installed rpm on original server are also present on new server
10373SME Contribs10.0CONFIRMEDMissing newline at end of config prevents conjob
10299SME Contribs10.0CONFIRMEDqpsmtpd user missing from /etc/shadow
9186SME Contribs10.0CONFIRMEDAFFA hardlinks using NFS storage for archives

Additional information

For details of performance, changelog etc see Affa:Additional information

References