Affa:Additional information

From SME Server
Revision as of 03:19, 18 October 2008 by Snoble (talk | contribs) (split page)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigationJump to search

Additional information related to Affa

Original wiki page

Performance

It is hard to predict how much time a backup job needs to complete. It depends on the number of files, the total file size, the file changes since last run, the network speed and not least on the CPU power, disk speed and RAM of the source and backup server. The following table of measured values will give you an idea of what you can expect.

Backup server Source server Data on source server Transferred Data Connection Compression Affa run time
2x3.2GHz Xeon
2 GB RAM, 1.5 TB RAID6 		2x3.2GHz Dual Core Xeon
4 GB RAM, RAID5, SME 7.1
Intranet Web Server + MySQL		 1.4 GB, 12,000 files 300 MB, 16 files Internet 2 Mbit yes 2 minutes
2x3.2GHz Xeon
2 GB RAM, 1.5 TB RAID6 		2x3.2GHz Dual Core Xeon
4 GB RAM, RAID5
SME 7.1 Mailserver		 43 GB, 410,000 files 140 MB, 2,700 files Internet 2 Mbit yes 10 minutes
2x2GHz Dual Core Xeon 5130
6 GB RAM, 1 TB RAID 5 		2x2GHz Dual Core Xeon 5130
6 GB RAM, 1 TB RAID 5
SME 7.1 File- and Mailserver		 125 GB, 98,000 files 3,2 GB, 3,000 files Gbit LAN no 25 minutes
2x2GHz Dual Core Xeon 5130
6 GB RAM, 874 GB RAID 5 		2x2GHz Dual Core Xeon 5130
6 GB RAM, 1 TB RAID 5
SME 7.1 File- and Mailserver		 125 GB, 98,000 files 3.5 GB, 2000 files Internet 2 Mbit yes 17 minutes
2x800MHz Pentium 3
1 GB RAM, 300 GB RAID1 		2x2.8GHz Xeon,
1GB RAM, 140 GB RAID5
SME 7.1 File- and Mailserver		 39 GB, 370,000 files 12 GB, 4,000 files 100Mbit LAN no 52 minutes
1xP4 2.4GHz
256 MB RAM
SME 7.1 		2xP4 1GHz
1 GB RAM
SME 6		 7.4 GB, 134,790 files 7.4 GB, 134,790 files 100Mbit LAN no 35 minutes

Note: The last action of a job run is to remove the oldest backup, e.g. if archive scheduled.11 exists and you have set the scheduledKeep property to 12, then it must be deleted. This can take a significantly long time, which increases the total job execution time.

A affa --rise run on a SME 7.2, 40 GB data, 411.500 files, 2x2.8 Xeon CPU, 1GB RAM, SW-RAID 5 with 4 U320 SCSI drives took 7 minutes including the final reboot.

Changelog

* Wed Apr 23 2008 Michael Weinberger 1.0.0-2
  Bug fix #4210, command "affa --show-schedule" executes postJobCommand
* Fri Apr 04 2008 Michael Weinberger 1.0.0-1
  Final release 
  man page updated 
* Fri Apr 04 2008 Michael Weinberger 0.10.0-2
  added --all option to --rise
* Mon Mar 10 2008 Michael Weinberger 0.10.0-1
  Increased release number
  Added 10 sec timeout in Net::DNS::Resolver->search
* Mon Mar 10 2008 Michael Weinberger 0.9.0-11
  Specfile fix
* Mon Mar 10 2008 Michael Weinberger 0.9.0-10
  Delete locally created watchdog file after remotecopy
  Bugfix 4035
  Added default AutoUnmount property to sample script
  Fix man page
  Do not delete md5sum in --create-backup-file when verify is skipped by user
  Set lock in --create-backup-file
* Sun Mar 09 2008 Michael Weinberger 0.9.0-9
  trap signals in --create-backup-file
* Sat Mar 08 2008 Michael Weinberger 0.9.0-8
  Permissions in /usr/lib/affa
* Sat Mar 08 2008 Michael Weinberger 0.9.0-7
  md5sum in --create-backup-file
* Fri Mar 07 2008 Michael Weinberger 0.9.0-6
  Added option --create-backup-file
  Using line number as error code in affaErrorExit()
  Fixed /etc/profile.d/affa.sh
* Mon Mar 03 2008 Michael Weinberger 0.9.0-5
  delete new style Dovecot's index files dovecot.index* 
  delete dovecot index files on remote server in --full-restore
* Fri Feb 29 2008 Michael Weinberger 0.9.0-4
  set localhost as default remoteHostName in /etc/e-smith/templates//etc/cron.d/affa/00jobs
* Wed Feb 27 2008 Michael Weinberger 0.9.0-3
  removed %post from spec file
* Sat Feb 02 2008 Michael Weinberger 0.9.0-2
  Bugfix: Report db was not deleted in deleteJob()
  don't use -q on ssh commands with Debug=yes
* Fri Feb 1 2008 Shad L. Lords <slords@mail.com> 0.9.0-1
  Fix specfile for building in new buildsystem
* Fri Jan 18 2008 Michael Weinberger
  Version 0.9.0
  Bugfix: run cronSetup() after job delete
  Throw error if nothing was backed up (empty archive)
  Added rsyncd support for backing up Windows/Cygwin
  Added option and property remoteOS. Which remoteOS=cgywin the 
  send key and revoke key functions use the right path to the 
  authorized_keys2 file and login as user Administrator
* Sat Dec 29 2007 Michael Weinberger
  Version 0.8.1
  Automount Bugfix [Bug 3165]: Only exit with error on mount failure when in backup run. Otherwise log the error and  continue.
* Thu Dec 20 2007 Michael Weinberger
  Version 0.8.0
  added bash completion script
  added option --kill JOB
  improved INT and TERM signal handling
  replaced all scp commands by rsync
  added poperty BandwidthLimit (unit KBytes/sec)
  each job write its own known hosts entry using ssh option HostKeyAliasOption
  affa --send-key w/o arguments sends the keys for all jobs
  removed Property StrictHostKeyChecking
  UPDATE FROM VERSIONS<8: run the command 'affa --send-key' to generate the known hosts entries for all jobs
* Wed Dec 12 2007 Michael Weinberger
  Version 0.7.0
  Modified logging: Job run logs to /var/log/affa/JOBNAME.log, all other to /var/log/affa/affa.log
  Bugfix: Check error status of --send-key  and --revoke-key
  New: Options --show-schedule
  Modified: Option --list-archives now accepts multiple args or none
  Fix: install watchdog on source server before every run not only scheduled, to avoid watchdog triggered if a non-scheduled run take to long time.
* Wed Nov 28 2007 Michael Weinberger
  Version 0.6.8
  Bugfix: Automount was done too late. Daily backup never ran.
* Wed Nov 21 2007 Michael Weinberger
  Version 0.6.7
  Bugfix: Size and No. of files were not shown in --status
  Improved deleting of directories
  added --check-connections option
  Increased width of job column in --stats table
* Sun Nov 11 2007 Michael Weinberger
  Version 0.6.6
  Property rsync--modify-window added
  Modified recursivly deleting directories
* Wed Nov 07 2007 Michael Weinberger
  Version 0.6.5
  ssh options added to scp commands
  added post-backup event
* Wed Aug 08 2007 Michael Weinberger
  Version 0.6.4
  delete dovcot index files after a --rise
* Thu Jul 05 2007 Michael Weinberger
  Version 0.6.3
  Bugfix: with RPMCheck=yes the affa-rpmlist was not found in the 
  archive. Wrong path to schedule.0 instead of scheduled.running. 
  Side effect of changes in version 0.5.0. [Bug 3128]
* Tue Jul 03 2007 Michael Weinberger
  Version 0.6.2
  Bugfix: Filesys::DiskFree requires $ENV{LANG}="en_US" [Bug 3119]
  Bug reported to bug-Filesys-DiskFree [at] rt.cpan.org
* Mon Jun 25 2007 Michael Weinberger 
  Version 0.6.1
  Fix 3080 was not applied.
* Mon Jun 18 2007 Michael Weinberger
  Version 0.6.0
  Replaced Filesys::DiskSpace by Filesys::DiskFree as the latter supports CIFS file system
  using Filesys::DiskFree->mount and ->device instead of scanning /proc/mounts
  mkdir RootDir only on job run to avoid making it in an unmounted mountpoint
* Mon Jun 18 2007 Michael Weinberger
  Version 0.5.2
  Sending ssh key failed. When grep returned an error
  the mv command was not executed. Using ';' instead of '&&' 
  for chaining commands. [Bug 3080]
  /root/.ssh/authorized_keys2: No such file or directory warning.
  Run a touch to guarantee, that the file exists. [Bug 3080]
  same issue with --revoke-key
* Thu Jun 14 2007 Michael Weinberger
  Version 0.5.1
  Bugfix automount: getStatus() unmounts the device while a job is running
  Status table: Last=never, if there are no archives at all
* Tue Jun 12 2007 Michael Weinberger
  Version 0.5.0
  New property AutomountOptions
  status table: grouped disabled jobs. Show Last='-' for disabled jobs instead of 'failed'
  unlink lockfile in getLock()
  Changes to improve robustness in case of network outage:
  - writing to a temporary archive scheduled.running and linkdest
    against the latest existing archive (usually scheduled.0). Then
    shift archives and rename scheduled.running to scheduled.0
  - scheduledKeep=1 is now allowed
  - also check connection in daily,weekly etc., not only in scheduled run
  - don't run daily, weekly, monthly or yearly, when scheduled.0 doesn't exist.
  new option --revoke-key
  added optional option --revoke-key to --delete-job
  avoid multiple adding public key to authorized_keys2 on source server
* Wed Jun 06 2007 Michael Weinberger
  Version 0.4.7
  use StrictHostKeyChecking=no in sendKeys()
* Tue Jun 05 2007 Michael Weinberger
  Version 0.4.6
  mailtest: Send also a testmail from the remotehost, when property Watchdog=yes
  removed useless code in mailTestWatchdogRemote()
* Mon Jun 04 2007 Michael Weinberger
  Version 0.4.5
  write job config file to the archive dir
* Thu May 31 2007 Michael Weinberger
  Version 0.4.4
  added property StrictHostKeyChecking (ssh option)
  string comparison instead of numric for $job{'doneYearly'} ne $thisYear etc.
* Wed May 30 2007 Michael Weinberger
  Version 0.4.3
  Minor bugfixes:
  calculation of RootDirFilesystemUsage in .AFFA-REPORT
  improved error handling with rsync status and df in DiskspaceWarn()
* Tue May 29 2007 Michael Weinberger
  Version 0.4.2
  Infinite loop in execPostJobCommand() if command could not be executed:
  Don't call execPostJobCommand() in affaErrorExit() if err==115
  RPMCheck property was ignored
  added Property chattyOnSuccess
  modified jobconfig-sample.pl to preserve 'doneDaily','doneWeekly','doneMonthly','doneYearly' and 'chattyOnSuccess'
  write error codes of affaErrorExit() to log
* Thu May 24 2007 Michael Weinberger
  Version 0.4.1
  fixed bug in disk usage calculation
* Mon May 21 2007 Michael Weinberger
  Version 0.4.0
  added option --rename-job
  changed syntax of --send-keys (!)
  added option --move-archive
* Sun May 20 2007 Michael Weinberger
  Version 0.3.3
  rebuild cronjobs after rise/undorise
  ignore job of own backup, when creating cronjobs (job appears after a rise run)
* Sun May 20 2007 Michael Weinberger
  Version 0.3.2
  man: added sshPort propperty
  chdir /tmp to avoid cwd warnings when the cwd disappears while running rise or undo rise
  bugfix: undorise() did not found own backup archive. Was searching for a wrong name
* Wed May 16 2007 Michael Weinberger
  Version 0.3.1 minor bugfixes
  checkCrossFS() did not work (used in --rise) 
  jobconfig-sample.pl: deleting record before setting props
  Perl errors with --status before a job run
* Thu May 10 2007 Michael Weinberger
  Version 0.3.0
  man page completed
  mark archives with indices > keep setting with '*' in --list-acrchive output
  Option --delete-job
  Option --cleanup 
  added --job=JOB alternative to --send-keys
* Wed May 09 2007 Michael Weinberger
  Version 0.2.0
  added --mailtest option
* Tue May 08 2007 Michael Weinberger
  Version 0.1.5
  improved --status output
  removed options --report and --send-report
  added option --show-archives
  added --csv for status and show-archives output in CSV format
  added property 'sshPort'
* Mon May 07 2007 Michael Weinberger
  Version 0.1.4
  don't install the remote watchdog, when remotehost is eq localhost
  improved check for remoteHostName eq localhost using DNS
  ssh -o PasswordAuthentication=no in checkConnection()
  added --full-restore
  missing check for HOSTNAME argument in --send-keys added
  improved error check
  prevent run of --rise of localhost from own backup
* Mon Apr 30 2007 Michael Weinberger
  Version 0.1.3
  modified 'use constant* syntax in watchdog script for compatibility with perl 5.6 on SME6
* Fri Apr 27 2007 Michael Weinberger
  Version 0.1.2
  Bugfix: Preserve of ethernet driver setting with --rise did not work.
  Also preserve NIC bonding.
* Mon Apr 23 2007 Michael Weinberger
  Version 0.1.1
  scheduledKeep must be>=2 for --link-dest
  set scheduledKeep to 2 if <2
  get lastrun date from affa-report rather than from report file
  added auto mount function
  added AutomountDevice and AutomountPoint to jobconfig-sample.pl
* Wed Apr 18 2007 Michael Weinberger
  Version 0.0.8
  dont die if report db does not exist
* Wed Apr 18 2007 Michael Weinberger
  Version 0.0.7
  run checkConnection() only for scheduled backups
  added Size and Disk usage information to --status
* Thu Apr 12 2007 Michael Weinberger
  Version 0.0.5
  fixed calculation of lastrun-now
* Thu Apr 12 2007 Michael Weinberger
  Version 0.0.4
  added --send-status plus templates 
  fixed format error of times in affa --status
  show 'failed', if lastrun is older 1 day in affa --status 
  fixed typo. default status=disabled (was disable)
* Fri Apr 06 2007 Michael Weinberger
  Version 0.0.3
  watchdog reminder was not deleted on soure
  wrong version mismatch list in  rpm compare
* Thu Apr 05 2007 Michael Weinberger
  added 'rsync--inplace' property
* Mon Apr 02 2007 Michael Weinberger
  initial release

Files

/etc/e-smith/events/actions/affa-make-cronjobs
/etc/e-smith/events/post-upgrade/S90affa-make-cronjobs
/etc/e-smith/templates/etc/cron.d/affa-status/00run
/etc/e-smith/templates/etc/cron.d/affa/00jobs
/etc/logrotate.d/affa
/etc/profile.d/affa.sh
/sbin/e-smith/affa
/sbin/e-smith/affa-rpmlist.sh
/usr/lib/affa/COPYING
/usr/lib/affa/jobconfig-cygwin-sample.pl
/usr/lib/affa/jobconfig-sample.pl
/usr/lib/affa/postJobCommand-sample.pl
/usr/lib/affa/preJobCommand-sample.pl
/usr/lib/affa/watchdog-mailtest.template
/usr/lib/affa/watchdog.template
/usr/man/man1/affa.1.gz

References

Retrieved from "https://wiki.koozali.org/index.php?title=Affa:Additional_information&oldid=11171"