Changes

From SME Server
Jump to navigationJump to search
5,633 bytes added ,  03:22, 2 November 2023
Line 3: Line 3:  
To view the status of contribs.org mirrors, look at http://mirror.contribs.org/mirrors.
 
To view the status of contribs.org mirrors, look at http://mirror.contribs.org/mirrors.
   −
===Accessing the contribs.org mirros===
+
===Accessing the contribs.org mirrors===
 
To access the contribs.org download mirrors, always use the URL http://mirror.contribs.org. The current releases can be found at http://mirror.contribs.org/smeserver/releases/, the contribs section can be found at: http://mirror.contribs.org/smeserver/contribs/  
 
To access the contribs.org download mirrors, always use the URL http://mirror.contribs.org. The current releases can be found at http://mirror.contribs.org/smeserver/releases/, the contribs section can be found at: http://mirror.contribs.org/smeserver/contribs/  
 
{{Tip box|If you use the mirrors.contribs.org URL you will automatically be redirected to a mirror that is current within the last 8 hours.}}
 
{{Tip box|If you use the mirrors.contribs.org URL you will automatically be redirected to a mirror that is current within the last 8 hours.}}
Line 12: Line 12:  
The obsolete directory accounts for 16Gb, and the testing directory which include the next version SME8 is 7Gb big.
 
The obsolete directory accounts for 16Gb, and the testing directory which include the next version SME8 is 7Gb big.
   −
If you are short in space you can easily save 16Gb with --exclude=obsolete/ in the EXCLUDE="" tag in etc/ftpsync.conf like this:
+
If you are short in space you can easily save 16Gb with --exclude=obsolete/ in the EXCLUDE="" tag in etc/ftpsync-smeserver.conf like this:
    
  EXCLUDE="--exclude=obsolete/"
 
  EXCLUDE="--exclude=obsolete/"
Line 30: Line 30:     
====Preparing your system====
 
====Preparing your system====
<ol><li>Create a storage location for mirror<br/>
+
<ol><li>Create a storage location for mirror<br />
 
<pre>mkdir -p {/path/to/your/data/store/}</pre>
 
<pre>mkdir -p {/path/to/your/data/store/}</pre>
 
</li><li>Create a new user to perform sync. If you are running SME Server you can create the user through the server-manager panel.
 
</li><li>Create a new user to perform sync. If you are running SME Server you can create the user through the server-manager panel.
 +
</li><li>Go to the new user's directory. On SME server this will be: <tt>/home/e-smith/files/users/{user}/</tt>
 +
cd {/path/to/user/dir/}
 
</li><li>Now it is time to download the ftpsync script and all files it requires:
 
</li><li>Now it is time to download the ftpsync script and all files it requires:
  wget http://contribs.org/ftpsync.tgz
+
  wget http://wiki.contribs.org/files/ftpsync.tgz
 
</li><li>Extract the tarball in users directory
 
</li><li>Extract the tarball in users directory
 
  tar zxof ftpsync.tgz
 
  tar zxof ftpsync.tgz
 
</li><li>Change the ownership of the directories to the new user
 
</li><li>Change the ownership of the directories to the new user
 
  chown -R {user} bin etc log locks .ssh {/path/to/your/data/store/}
 
  chown -R {user} bin etc log locks .ssh {/path/to/your/data/store/}
</li><li>Now we have installed and set things up as is required but we will need to update the configuration file to point to the storage location of the data (TO) in the config file (etc/ftpsync-smeserver.conf). Use your favorite text editor for it.</li></ol>
+
</li><li>Now we have installed and set things up as is required but we will need to update the configuration file to point to the storage location of the data (TO) in the config file (etc/ftpsync-smeserver.conf). Use your favorite text editor for it.</li>
 +
TO="/path/to/your/data/store/"
 +
<li>'''Only''' when you are setting your server up as a [[#Configuring for pull|pull-mirror]], set the ''RSYNC_HOST'' value in the config file (etc/ftpsync-smeserver.conf) to (this is the default setting - disable for push) :<pre>RSYNC_HOST=mirror.canada.pialasse.com</pre>This because of the recent unreliability of ibiblio.org (see: [[bugzilla:7360]]).</li></ol>
    
====Testing your setup====
 
====Testing your setup====
 
<ol><li>Now it is time to perform the initial sync (and test that script does what it needs to)
 
<ol><li>Now it is time to perform the initial sync (and test that script does what it needs to)
 +
{{Note box| This may take a long time depending on the speed of your connection}}
 
<pre>su - {user} -s /bin/bash
 
<pre>su - {user} -s /bin/bash
 
~/bin/ftpsync sync:archive:smeserver</pre>
 
~/bin/ftpsync sync:archive:smeserver</pre>
</li><li>Now check heck the logs to see if there are any errors. Since the initial sync will take a lot of time you can best do this in a second terminal window:
+
</li><li>Now check the logs to see if there are any errors. Since the initial sync will take a lot of time you can best do this in a second terminal window:
 
  cd ~/log
 
  cd ~/log
 
  cat rsync-ftpsync-smeserver.error.0
 
  cat rsync-ftpsync-smeserver.error.0
Line 103: Line 108:  
=====Configuring for pull=====
 
=====Configuring for pull=====
 
{{Note box|We prefer you configure your mirror to be setup as a push mirror, but if you can not do so or have other reasons for not doing so you can also configure your mirror to pull.}}
 
{{Note box|We prefer you configure your mirror to be setup as a push mirror, but if you can not do so or have other reasons for not doing so you can also configure your mirror to pull.}}
 +
{{Note box| See Bug list at bottom of page re error on ftpsync pull request when using this version of ftpsync, see Bugzilla 11754}}
 +
 
Configuring for a pull based mirror is easy. Just schedule a cron job to run every 2 hours that does the exact same sync command you do to get the mirror in the first place, you can add a comment like in the example below:
 
Configuring for a pull based mirror is easy. Just schedule a cron job to run every 2 hours that does the exact same sync command you do to get the mirror in the first place, you can add a comment like in the example below:
   −
  1 */2  * * *  {user} ~/bin/ftpsync
+
  1 */2  * * *  {user} ~/bin/ftpsync sync:archive:smeserver
    
====Advertising your mirror====
 
====Advertising your mirror====
Line 120: Line 127:     
===Configuration options===
 
===Configuration options===
The ftpsync configuration file can be found in ~/etc/ftpsync.conf. The ftpsync configuration file has a number of options you can configure. You might have already seen some of them when you had to adjust the storage location in the configuration process.
+
The ftpsync configuration file can be found in ~/etc/ftpsync-smeserver.conf. The ftpsync configuration file has a number of options you can configure. You might have already seen some of them when you had to adjust the storage location in the configuration process.
 
The configuration file is well documented but we will discuss some of the features here.
 
The configuration file is well documented but we will discuss some of the features here.
 
{{Incomplete}}
 
{{Incomplete}}
 +
 +
 +
===Configure your mirror as hub===
 +
 +
this is intented to do on an already synced mirror in push mode.
 +
 +
==== setting up rsync as server====
 +
as root.
 +
 +
 +
open /etc/xinetd.d/rsync and remove the disabled line
 +
vim /etc/xinetd.d/rsync
 +
 +
create a rsyncd.conf file
 +
vim /etc/rsyncd.conf
 +
 +
then add this
 +
 +
# rsync.conf
 +
uid = {USEROWNING THE MIRROR FILES}
 +
gid = {GROUP OWNING THE MIRROR FILES}
 +
use chroot = yes
 +
max connections = 5
 +
timeout = 300
 +
read only = true
 +
strict modes = true
 +
transfer logging = true
 +
dont compress = *.gz *.tgz *.zip *.z *.rpm *.deb *.iso *.bz2 *.tbz
 +
 +
[smeserver]
 +
    path = /home/e-smith/files/ibays/{IBAY}/html
 +
    comment = SME Server Mirror
 +
    hosts allow = {YOUR CLIENT IP}
 +
 +
install xinetd
 +
yum install xinetd
 +
 +
configure as services
 +
ln -s /etc/rc.d/init.d/e-smith-service /etc/rc7.d/S91xinetd
 +
config set xinetd service status enabled
 +
config set rsyncd service TCPPort 873 access public status enabled
 +
signal-event remoteaccess-update
 +
service xinetd start
 +
 +
 +
configure hosts.allow
 +
 +
mkdir -p /etc/e-smith/templates-custom/etc/hosts.allow/
 +
echo "rsync: ALL" > /etc/e-smith/templates-custom/etc/hosts.allow/rsync
 +
signal-event remoteaccess-update
 +
 +
you can browse your server like this
 +
rsync --list-only rsync://YourIP
 +
 +
you can download from your server like this
 +
rsync -avz  YourIP::smeserver YourLocalFolder
 +
 +
==== create a rsa file ====
 +
 +
as user responsible of ftpsync.
 +
 +
  ssh-keygen -qt rsa -b 1024 -C "contribs-push@{MY MIRROR}" -f ~/.ssh/contribs-push\@{MY MIRROR}.rsa
 +
 +
then to protect the key and limit it to one usage and only one IP. You need to change {MY MIRROR} and {MY HUB IP }
 +
 +
echo "no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty,command=\"~/bin/ftpsync\",from=\"{MY HUB IP }\" "|cat - ~/.ssh/contribs-push\@{MY MIRROR}.rsa.pub > ~/.ssh/contribs-push.tmp && mv ~/.ssh/contribs-push.tmp ~/.ssh/contribs-push\@{MY MIRROR}.rsa.pub
 +
 +
finally protect your keys
 +
chmod 0600 ~/.ssh/contribs-push\@{MY MIRROR}.rs*
 +
 +
never give your private key, only send your  public (.pub) to the downstream mirror.
 +
 +
==== setting up ftpsync-smeserver.conf====
 +
 +
as user responsible of ftpsync.
 +
 +
 +
this step will allow your ftpsync to know that he need to trigger another server
 +
 +
edit ~/etc/ftpsync-smeserver.conf
 +
 +
vim ~/etc/ftpsync-smeserver.conf
 +
 +
under the line #HUB=false, insert
 +
HUB=true
 +
 +
==== setting up runmirrors-smeserver.conf====
 +
 +
as user responsible of ftpsync.
 +
 +
 +
vim ~/etc/runmirrors-smeserver.conf
 +
 +
then comment out the following line and adapt it to the path of you private key
 +
KEYFILE=.ssh/contribs-push\@{MY MIRROR}.rsa
 +
 +
==== setting up runmirrors-smeserver.mirror====
 +
 +
as user responsible of ftpsync.
 +
 +
 +
edit the file
 +
vim ~/etc/runmirrors-smeserver.mirror
 +
 +
add the following at the end of the file
 +
 +
mhop {NAME FOR LOG PURPOSE} {YOUR CLIENT ADRESS} {YOUR CLIENT USER} -p {YOUR CLIENT SSH PORT}
 +
 +
 +
===Configure your mirror under another hub===
 +
 +
follow : http://wiki.contribs.org/Mirrors#How_to_become_a_mirror_site.3F
 +
 +
except you will have to import your own public key  (generated just above)
 +
 +
then you will have to tunes :
 +
 +
vim ~/etc/ftpsync-smeserver.conf
 +
 +
change RSYNC_HOST= to point to your hub mirror
 +
 +
RSYNC_HOST={HUB MIRROR ADRESSE}
 +
 +
tada! you are done!
 +
 +
=== Current SME Server contribs.org Mirror Tree ===
 +
{| class="wikitable"
 +
|+
 +
| rowspan="11" |koozali.org mirrors
 +
| rowspan="11" |mirror.canada.pialasse.com
 +
Rsync, Push
 +
|smeserver.bhs.mirrors.ovh.net
 +
Rsync / 1GPs /pull %36h
 +
|
 +
|-
 +
|distro.ibiblio.org
 +
Rsync, Push
 +
|ftp.iinet.net.au
 +
mirror.internode.on.net
 +
 +
ftp.icm.edu.pl
 +
|-
 +
|mirror.pialasse.com
 +
Push
 +
|
 +
|-
 +
|sme-mirror.tw.co.nz
 +
Pull
 +
|
 +
|-
 +
|sme-mirror.firewall-services.com
 +
Pull
 +
|
 +
|-
 +
|ibsgaarden.dk
 +
Push
 +
|
 +
|-
 +
|mirrors.rbx.opencare.nl
 +
Push
 +
|
 +
|-
 +
|ftp.nluug.nl / ftp.vim.org
 +
Pull
 +
|
 +
|-
 +
|ftp.icm.edu.pl
 +
 +
Pull
 +
|
 +
|-
 +
|mirrors.mab974.re
 +
Pull
 +
|
 +
|-
 +
|www.mirrorservice.org
 +
Rsync, Pull
 +
|
 +
|}
 +
 +
====RSYNC access====
 +
# ibiblio : rsync://distro.ibiblio.org/smeserver/releases/ ( rsync -avv --stats distro.ibiblio.org::smeserver/releases . )
 +
# mirror.canada.pialasse.com:  rsync -avv mirror.canada.pialasse.com::smeserver/releases .
 +
# mirrorservice.org : rsync[http://rsync.mirrorservice.org/sites/mirror.contribs.org/smeserver/releases/ //rsync.mirrorservice.org/sites/mirror.contribs.org/smeserver/releases/] (rsync -avv --stats rsync.mirrorservice.org::mirror.contribs.org/smeserver/releases/ . )
 +
 +
====FTP access====
 +
# ftp://distro.ibiblio.org/smeserver/releases/
 +
 +
=== Bugs ===
 +
Please raise bugs under the SME-Contribs section in [http://bugs.contribs.org/enter_bug.cgi bugzilla]
 +
and select the Mirrors component or use {{BugzillaFileBug|product=SME%20Contribs|component=Mirrors|title=this link}}
 +
{{#bugzilla:columns=id,product,version,status,summary |sort=id |order=desc |component=Mirrors |noresultsmessage="No open bugs found."}}
 +
 +
 +
[[Category:Howto]]
 +
[[Category:Development Tools]]

Navigation menu