Changes

From SME Server
Jump to navigationJump to search

Backup of ESXi Virtual Machines using Affa (view source)

Revision as of 21:04, 2 November 2013

98 bytes added ,  21:04, 2 November 2013
→‎Configure the DAG repository
Line 2: Line 2:     
=== Introduction ===
 
=== Introduction ===
−
As of version 2, Affa supports hot backing up virtual machines running on the VMWare ESXi server. It uses the VMware Infrastructure Perl Toolkit to communicate with the EXSi and rsync over ssh to transfer the data. Before the backup starts, the state of the running VM is freezed by creating a snapshot of all its disks. After the freezed virtual disk files were backed up, the snapshot is deleted and the config points to the just backed up disks again. Then, as a last step the config files are backed up.
     −
A restore of a VM can be done by simply copying back all archived files to the original location on the ESXi host. It is also possible to copy the files to a different directory (or to another ESXi server) and add the vmx file to the inventory.
+
[[Affa]] supports hot backing up virtual machines running on the VMWare ESXi server. It uses the VMware Infrastructure Perl Toolkit to communicate with the EXSi and rsync over ssh to transfer the data. Before the backup starts, the state of the running VM is freezed by creating a snapshot of all its disks. After the freezed virtual disk files were backed up, the snapshot is deleted and the config points to the just backed up disks again. Then, as a last step the config files are backed up.
   −
{{Note box|This document is applicable to ESX3i with an USB boot device. It will most likely not work with other type boot devices}}
+
A restore of a VM can be done by simply copying back all archived files to the original location on the ESXi host. It is also possible to copy the files to a different directory (or to another ESXi server) and add the vmx file to the inventory. Alternatively with SambaShare=yes the VMWare Converter tool can be used.
   −
{{Warning box|As Affa Version 2 is in a BETA state, errors in design and/or implementation cannot be completely eliminated. Therefore take precautions to avoid negative impacts to your system. In particular do not use it on production systems.}}
+
 
 +
{{Note box|This document is applicable to ESX3i with an USB boot device. It will most likely not work with other type of boot devices}}
 +
 
 +
{{Warning box|This software comes 'as is' without any warranty.  Errors in design and/or implementation cannot be completely eliminated. Therefore take precautions to avoid negative impacts to your system.}}
    
=== Preparing the Affa Server ===
 
=== Preparing the Affa Server ===
−
==== Install the VMware Infrastructure (VI) Perl Toolkit ====
+
==== Affa Installation ====
−
Download the VI Perl Toolkit tarball from the VMWare web site: http://www.vmware.com/support/developer/viperltoolkit/
+
Install the Affa program as descibed here: [[Affa#Installation_or_Update]]
 +
 
 +
==== VMware Infrastructure (VI) Perl Toolkit Installation ====
 +
===== Configure the DAG repository =====
 +
You need to activate the [[Dag]] reposity before installing this contrib.<br />
 +
 
 +
see [[dag|dag repository]] <br />
 +
 
    
Login as root and run:
 
Login as root and run:
 
  /usr/bin/yum install openssl-devel
 
  /usr/bin/yum install openssl-devel
 
  /usr/bin/yum --enable=dag install perl-XML-SAX
 
  /usr/bin/yum --enable=dag install perl-XML-SAX
 +
 +
===== Install the VMware Infrastructure (VI) Perl Toolkit =====
 +
Download the VI Perl Toolkit tarball from the VMWare web site: http://www.vmware.com/support/developer/viperltoolkit/
 
  tar xzf VMware-VIPerl-1.6.0-104313.i386.tar.gz
 
  tar xzf VMware-VIPerl-1.6.0-104313.i386.tar.gz
 
  cd vmware-viperl-distrib
 
  cd vmware-viperl-distrib
 +
Edit the vmware-install.pl script and comment out these lines
 +
  4738 #      if ( -d '/proc/xen' ) {
 +
  4739 #        error('You cannot install ' .
 +
  4740 #              vmware_product_name() .
 +
  4741 #              ' on a system running a xen kernel.');
 +
  4742 #      }
 +
Run the script
 
  ./vmware-install.pl
 
  ./vmware-install.pl
 
Ignore this warning:
 
Ignore this warning:
Line 26: Line 45:  
XML::NamespaceSupport
 
XML::NamespaceSupport
   −
==== Install Affa Version 2 beta ====
+
 
−
Install the '''latest''' Affa 2.x beta version. Download the RPM from http://mirror.contribs.org/smeserver/contribs/michaelw/sme7/Affa2
  −
/usr/bin/yum --enable=smecontribs localinstall smeserver-affa-2.0.0-beta*.noarch.rpm
      
=== Preparing the ESXi Host for use with Affa ===
 
=== Preparing the ESXi Host for use with Affa ===
−
To make the ESXi server ready to cooperate with Affa you must install the rsync program, enable the ssh service, configure passwordless ssh login and add a user for communication using the VMware Infrastructure (VI) Perl Toolkit.
+
To make the ESXi server ready to cooperate with Affa you must install the rsync program, enable the ssh service, configure passwordless ssh login and add a user for API communication using the VMware Infrastructure (VI) Perl Toolkit.
 
==== Adding an ESXi User with restricted Permissions ====
 
==== Adding an ESXi User with restricted Permissions ====
 
Run the Virtual Infrastructure Client and logon to the ESXi host.
 
Run the Virtual Infrastructure Client and logon to the ESXi host.
Line 49: Line 66:     
==== Configuring SSH and RSYNC on the ESX3i Host ====
 
==== Configuring SSH and RSYNC on the ESX3i Host ====
−
By default the SSH service is disabled and rsync is not installed. ESXi uses Dropbear rather than OpenSSL as the SSH Server. The installed Dropbear Version 0.49 has a bug that causes an incorrect return status of the executed command. Therefore it must be updated to Version 0.51. This version fixes the problem with wrong exit codes. Only with Dropbear Version 0.51 Affa can detect failures of rsync commands.
+
On ESXi the SSH service Dropbear is installed but disabled. To use rsync the SSH service must be enabled and the rsync program must be installed. To enable passwordless login a .ssh directory for storing the keys must be created.
   −
To set up ESXi for use with Affa we must:
+
A script <tt>affa-setup.sh</tt> is provided that does this:
 
* enable SSH service by configuring inetd.conf and restarting the inetd service
 
* enable SSH service by configuring inetd.conf and restarting the inetd service
 
* install rsync (statically linked binary)
 
* install rsync (statically linked binary)
−
* update Dropbear to version 0.51
   
* create a /root home directory  with a .ssh sub directory to store the public keys for password-less login
 
* create a /root home directory  with a .ssh sub directory to store the public keys for password-less login
 
* link the .ssh directory to the non-volatile USB device
 
* link the .ssh directory to the non-volatile USB device
 
* add a command to /etc/rc.local that executes all the steps above at boot time
 
* add a command to /etc/rc.local that executes all the steps above at boot time
   −
The following script does this all:
+
===== Download and install the script =====
−
<pre>
  −
# enable ssh service
  −
sed -e 's/^#\(ssh.*\)/\1/' < /etc/inetd.conf > /etc/inetd.conf.affa
  −
mv -f /etc/inetd.conf.affa /etc/inetd.conf
  −
 
  −
# reload inetd configuration
  −
kill -HUP `ps | grep inetd | sed -e 's/ .*//'`
  −
 
  −
# install Dropbear 0.51
  −
rm -f /sbin/dropbearmulti; cp -a /bootbank/bexi/dropbearmulti /sbin/dropbearmulti
  −
ln -fs /sbin/dropbearmulti /bin/ssh
  −
 
  −
# install sftp-server (needed for FUSE sshfs mount)
  −
cp -a /bootbank/bexi/sftp-server-static /usr/libexec/sftp-server
  −
 
  −
# install rsync 3.0.4 statically linked
  −
cp -a /bootbank/bexi/rsync-static /bin/rsync
  −
 
  −
# root home dir for storing public keys
  −
mkdir -p /bootbank/bexi/root/.ssh
  −
touch /bootbank/bexi/root/.ssh/authorized_keys
  −
touch /bootbank/bexi/root/.ssh/known_hosts
  −
 
  −
# link .ssh dir to non-volatile USB location
  −
mkdir -p /root
  −
ln -fs /bootbank/bexi/root/.ssh /root/
  −
 
  −
# set new home dir for user root
  −
sed -e 's#^\(root:.*\)\(:/:\)\(.*\)#\1:/root:\3#' < /etc/passwd > /etc/passwd.affa
  −
mv -f /etc/passwd.affa /etc/passwd
  −
 
  −
# add execution of this script to rc.local
  −
grep -q "/bootbank/bexi/affa-setup.sh" < /etc/rc.local || echo "/bootbank/bexi/affa-setup.sh" >> /etc/rc.local
  −
</pre>
  −
 
   
On the ESXi Server console hit Alt-F1 to get the service shell. Now blind-type the word '''unsupported''' to unlock the shell.  
 
On the ESXi Server console hit Alt-F1 to get the service shell. Now blind-type the word '''unsupported''' to unlock the shell.  
   Line 101: Line 82:  
[[Image:Affa_esxi_service_console.gif‎]]
 
[[Image:Affa_esxi_service_console.gif‎]]
   −
Download the tarball. The tarball contains all binaries and the script.  
+
Download the tarball. The tarball contains the rsync binary and the script.  
 
  cd /bootbank
 
  cd /bootbank
−
  wget http://mirror.contribs.org/smeserver/contribs/michaelw/sme7/Affa2/affa-esxi-setup-01.tgz  
+
  wget <nowiki>http://mirror.contribs.org/smeserver/contribs/michaelw/sme7/Affa2/affa-esxi-setup-02.tgz</nowiki>
 
Verify the download
 
Verify the download
−
   wget http://mirror.contribs.org/smeserver/contribs/michaelw/sme7/Affa2/affa-esxi-setup-01.tgz.md5sum  
+
   wget <nowiki>http://mirror.contribs.org/smeserver/contribs/michaelw/sme7/Affa2/affa-esxi-setup-02.tgz.md5sum</nowiki>
−
   md5sum -c affa-esxi-setup-01.tgz.md5sum
+
   md5sum -c affa-esxi-setup-02.tgz.md5sum
 
Unpack it
 
Unpack it
−
  tar xzf affa-esxi-setup-01.tgz  
+
  tar xzf affa-esxi-setup-02.tgz  
−
  rm affa-esxi-setup-01.tgz*  
+
  rm affa-esxi-setup-02.tgz*  
 
and run the script
 
and run the script
 
  ./bexi/affa-setup.sh
 
  ./bexi/affa-setup.sh
Line 116: Line 97:     
=== Setting up an Affa job ===
 
=== Setting up an Affa job ===
−
The following assumes, that you are already familiar with configuring Affa and only focus on the ESXi specific parts.
+
The following assumes, that you are already familiar with configuring [[Affa]] and only focus on the ESXi specific parts.
    
Assume you want to backup the virtual machine named 'myvm' on ESXi server with IP 10.200.48.5.
 
Assume you want to backup the virtual machine named 'myvm' on ESXi server with IP 10.200.48.5.
   −
1. log into the 'affabox' and copy the config helper script
+
* log into the 'affabox' and copy the config helper script
 
  cp -a /usr/lib/affa/jobconfig-esxi-sample.pl /root/esxi-myvm-job.pl
 
  cp -a /usr/lib/affa/jobconfig-esxi-sample.pl /root/esxi-myvm-job.pl
−
2. edit <tt>/root/esxi-myvm-job.pl</tt> and set
+
* edit <tt>/root/esxi-myvm-job.pl</tt> and set
 
  my $jobname='esxi-myvm';
 
  my $jobname='esxi-myvm';
 
and
 
and
 
  'remoteHostName‘=>'10.200.48.5',
 
  'remoteHostName‘=>'10.200.48.5',
−
3. tell Affa that this job backups a ESXi virtual machine
+
* tell Affa that this job backups a ESXi virtual machine
 
  'ESXi' => 'yes',
 
  'ESXi' => 'yes',
−
4. set the VM name
+
* set the VM name
 
  'ESXiVMName' => 'myvm',
 
  'ESXiVMName' => 'myvm',
−
5. set the username and password (this is the ESXi User with restricted permissions created above)
+
* set the username and password (this is the ESXi User with restricted permissions created above)
 
  'ESXiUsername' => 'affa',
 
  'ESXiUsername' => 'affa',
 
  'ESXiPassword' => 'secret',
 
  'ESXiPassword' => 'secret',
−
6. save the script, then run it
+
* configure virtual disk files to be compressed and chunked
 +
'chunkFiles' => '*.vmdk',
 +
* save the script, then run it
 
  /root/esxi-myvm-job.pl
 
  /root/esxi-myvm-job.pl
−
7. send the public key
+
* send the public key
 
  affa --send-key esxi-myvm
 
  affa --send-key esxi-myvm
−
8. run the job manually
+
* run the job manually
 
  affa --run esxi-myvm
 
  affa --run esxi-myvm
   Line 147: Line 130:  
=== Uninstall ===
 
=== Uninstall ===
 
SSH login as root on the ESXi host and run these commands '''carefully''':
 
SSH login as root on the ESXi host and run these commands '''carefully''':
−
  grep -v "/bootbank/bexi/affa-setup.sh" < /etc/rc.local >/etc/rc.local.affa; mv -f /etc/rc.local.affa /etc/rc.local
+
  grep -v "#BEXI" < /etc/rc.local >/etc/rc.local.affa; mv -f /etc/rc.local.affa /etc/rc.local
 
  sed -e 's#^\(root:.*\)\(:/root:\)\(.*\)#\1:/:\3#' < /etc/passwd > /etc/passwd.affa; mv -f /etc/passwd.affa /etc/passwd
 
  sed -e 's#^\(root:.*\)\(:/root:\)\(.*\)#\1:/:\3#' < /etc/passwd > /etc/passwd.affa; mv -f /etc/passwd.affa /etc/passwd
 
  sed -e 's/^\(ssh.*\)/#\1/' < /etc/inetd.conf > /etc/inetd.conf.affa; mv -f /etc/inetd.conf.affa /etc/inetd.conf
 
  sed -e 's/^\(ssh.*\)/#\1/' < /etc/inetd.conf > /etc/inetd.conf.affa; mv -f /etc/inetd.conf.affa /etc/inetd.conf
 
  kill -HUP `ps | grep inetd | sed -e 's/ .*//'`
 
  kill -HUP `ps | grep inetd | sed -e 's/ .*//'`
−
  rm -rf /bin/rsync /usr/libexec/sftp-server /root /bootbank/bexi
+
  rm -rf /bin/rsync /root /bootbank/bexi
 
  tar -C / -xzf /bootbank/environ.tgz sbin/dropbearmulti
 
  tar -C / -xzf /bootbank/environ.tgz sbin/dropbearmulti
    
=== Addtional Information ===
 
=== Addtional Information ===
−
===== How to make the Dropbearmulti 0.51 Binary and the statically linked Rsync Binary =====
  −
If you want to make the binaries by yourself, run the following on a Centos4/SME7.3 box with devel RPMs installed:
     −
'''Dropbear 0.51'''
     −
1. download <tt>dropbear-0.51.tar.gz</tt> from http://matt.ucc.asn.au/dropbear/
     −
2. unpack the tarball
  −
tar xzf dropbear-0.51.tar.gz
  −
cd dropbear-0.51
     −
3. edit the file <tt>options.h</tt> and comment out line 40
+
===== How to update the ESXi System Image =====
−
  /*#define NON_INETD_MODE*/
+
Shutdown the ESXi server. Remove the USB Memory stick and plug it into a linux computer.
 +
run the <tt>dmesg</tt> command to find out the device the memory stick is assigned to. The following example assumes that it is /dev/sdf. Replace is by the correct device.
 +
mkdir -p /mnt/esxi
 +
mount -t vfat /dev/sdf5 /mnt/esxi
 +
Save local.tgz and the bexi/ directory to the local machine
 +
cp -a /mnt/esxi/local.tgz /mnt/esxi/bexi .
 +
  umount /mnt/esxi
 +
Download the ISO from the [http://www.vmware.com/download/esxi/ VMWare website] and extract the image.
   −
4. configure and compile
+
mount -o loop VMware-VMvisor-InstallerCD*iso /mnt/esxi/
−
  ./configure --disable-zlib --disable-openpty
+
mkdir -p /tmp/esxi
−
  make PROGRAMS="dropbear dbclient scp dropbearkey dropbearconvert" MULTI=1 STATIC=0 SCPPROGRESS=1
+
tar -xvzf /mnt/esxi/install.tgz -C /tmp/esxi/
−
  strip dropbearmulti
+
umount /mnt/esxi
 +
cd /tmp/esxi/usr/lib/vmware/installer/
 +
  bzip2 -d VMware-VMvisor-big*.dd.bz2
 +
Write the new image to the memory stick. Make sure that you use the correct target device of=...
 +
dd if=VMware-VMvisor-big-3.5.0_Update_3-123629.i386.dd of=/dev/sdf
 +
rm -rf /tmp/esxi
 +
Copy local.tgz and the bexi/ directory to the memory stick.
 +
mount -t vfat /dev/sdf5 /mnt/esxi
 +
  cp -a local.tgz bexi /mnt/esxi
 +
  umount /mnt/esxi
 +
Plug the memory stick into the ESXi server and power on.
    +
Note: Alternatively you can install the new ESXi image on a second memory stick and then remote copy local.tgz and bexi/ from the running ESXi. Then shut down the ESXi server, replace the stick and power on. This minimizes the down time.
   −
'''Rsync 3.0.4 statically linked'''
+
===== How to make a statically linked Rsync Binary =====
−
 
+
''(This is just for information. A pre-build Rsync static is included in the affa-esxi-setup-*.tgz tarball)''
−
1. download the source from http://rsync.samba.org/
+
* download the source from http://rsync.samba.org/
−
 
+
* unpack the tarball
−
2. unpack the tarball
   
  tar xzf rsync-3.0.4.tar.gz
 
  tar xzf rsync-3.0.4.tar.gz
 
  cd rsync-3.0.4
 
  cd rsync-3.0.4
−
3. build the statically linked binary
+
* build the statically linked binary
 
  make CFLAGS="-static" EXEEXT="-static"
 
  make CFLAGS="-static" EXEEXT="-static"
 
  strip rsync-static
 
  strip rsync-static
   −
 
+
=== References ===
 +
Wiki article [[Affa]]
    
[[Category:Howto]]
 
[[Category:Howto]]
 
[[Category: Backup]]
 
[[Category: Backup]]
 +
[[Category: Virtualisation]]
Retrieved from "https://wiki.koozali.org/Special:MobileDiff/11517...22070"

Navigation menu