Difference between revisions of "Rclone"

From SME Server
Jump to navigationJump to search
 
(17 intermediate revisions by 2 users not shown)
Line 19: Line 19:
 
===Maintainer===
 
===Maintainer===
 
<!-- here you need to file your username and name -->
 
<!-- here you need to file your username and name -->
−
[[User:brianr|Brian Read]]  
+
[[User:brianr|Brian Read]]
  
 
=== Version ===
 
=== Version ===
 
<!-- keep this first element as is, you can add some if needed -->
 
<!-- keep this first element as is, you can add some if needed -->
−
{{#smeversion: {{#var:smecontribname}} }}Not yet built into an rpm.  Requires command level knowledge to install and run.
+
<!-- {{#smeversion: {{#var:smecontribname}} }}Not built into an rpm.  -->
 +
 
 +
Requires command level knowledge to install and run.
  
 
=== Description ===
 
=== Description ===
  
−
[https://rclone.org/ Rclone] is a program written in go to allow local files to be saved to one of many cloud servers.  It tries to emulate rsync.
+
[https://rclone.org/ Rclone] is a program written in Go to allow local files to be saved to one of many cloud servers.  It tries to emulate rsync.
  
 
Current cloud providers supported are:
 
Current cloud providers supported are:
  
−
    Amazon Drive
+
 
−
    Amazon S3
+
* Alibaba Cloud (Aliyun) Object Storage System (OSS)
−
    Backblaze B2
+
* Amazon Drive   (See note)
−
    Box
+
* Amazon S3
−
    Ceph
+
* Backblaze B2
−
    DigitalOcean Spaces
+
* Box
−
    Dreamhost
+
* Ceph
−
    Dropbox
+
* DigitalOcean Spaces
−
    FTP
+
* Dreamhost
−
    Google Cloud Storage
+
* Dropbox
−
    Google Drive
+
* FTP
−
    HTTP
+
* Google Cloud Storage
−
    Hubic
+
* Google Drive
−
    Memset Memstore
+
* HTTP
−
    Microsoft Azure Blob Storage
+
* Hubic
−
    Microsoft OneDrive
+
* Jottacloud
−
    Minio
+
* IBM COS S3
−
    Nextloud
+
* Koofr
−
    OVH
+
* Memset Memstore
−
    Openstack Swift
+
* Mega
−
    Oracle Cloud Storage
+
* Microsoft Azure Blob Storage
−
    Ownloud
+
* Microsoft OneDrive
−
    pCloud
+
* Minio
−
    put.io
+
* Nextcloud
−
    QingStor
+
* OVH
−
    Rackspace Cloud Files
+
* OpenDrive
−
    SFTP
+
* Openstack Swift
−
    Wasabi
+
* Oracle Cloud Storage
−
    WebDAV
+
* ownCloud
−
    Yandex Disk
+
* pCloud
−
    The local filesystem
+
* put.io
−
 
+
* QingStor
 +
* Rackspace Cloud Files
 +
* Scaleway
 +
* SFTP
 +
* Wasabi
 +
* WebDAV
 +
* Yandex Disk
 +
* The local filesystem
 
I have only tested it against Dropbox.
 
I have only tested it against Dropbox.
  
 
=== Installation ===
 
=== Installation ===
  
−
Rclone can be installed as follows: (https://rclone.org/install/)
+
Rclone is made available as a statically linked executible and can be installed as follows: (https://rclone.org/install/)
  
−
<code>
+
<pre>
 
curl -O https://downloads.rclone.org/rclone-current-linux-amd64.zip
 
curl -O https://downloads.rclone.org/rclone-current-linux-amd64.zip
−
 
 
unzip rclone-current-linux-amd64.zip
 
unzip rclone-current-linux-amd64.zip
−
 
 
cd rclone-*-linux-amd64
 
cd rclone-*-linux-amd64
−
 
 
sudo cp rclone /usr/bin/
 
sudo cp rclone /usr/bin/
−
 
 
sudo chown root:root /usr/bin/rclone
 
sudo chown root:root /usr/bin/rclone
−
 
 
sudo chmod 755 /usr/bin/rclone
 
sudo chmod 755 /usr/bin/rclone
−
 
+
</pre>
−
</code>
 
  
 
Note that this loads the 64 bit version, other versions are available.
 
Note that this loads the 64 bit version, other versions are available.
Line 88: Line 91:
 
Then you can run the configuration process:
 
Then you can run the configuration process:
  
−
<code>rclone config</code>
+
<pre>rclone config</pre>
  
 
Choose your cloud provider and follow the instructions for a headless server.
 
Choose your cloud provider and follow the instructions for a headless server.
Line 98: Line 101:
 
Here is the shell script that I developed and tested to see if it would work:
 
Here is the shell script that I developed and tested to see if it would work:
  
−
<code>
+
<pre>
 
!/bin/sh
 
!/bin/sh
−
 
 
signal-event pre-backup
 
signal-event pre-backup
−
 
 
cd /
 
cd /
−
 
 
rclone mkdir smeserver-dropbox:backup
 
rclone mkdir smeserver-dropbox:backup
−
 
 
rclone mkdir smeserver-dropbox:backup/smeserver
 
rclone mkdir smeserver-dropbox:backup/smeserver
−
 
 
rclone mkdir smeserver-dropbox:backup/smeserver/etc/e-smith/templates-user-custom
 
rclone mkdir smeserver-dropbox:backup/smeserver/etc/e-smith/templates-user-custom
−
 
 
rclone mkdir smeserver-dropbox:backup/smeserver/etc/e-smith/templates-custom
 
rclone mkdir smeserver-dropbox:backup/smeserver/etc/e-smith/templates-custom
 +
rclone mkdir smeserver-dropbox:backup/smeserver/home
 +
rclone mkdir smeserver-dropbox:backup/smeserver/ssh
 +
rclone mkdir smeserver-dropbox:backup/smeserver/root
 +
rclone copy -v  / smeserver-dropbox:backup/smeserver --files-from backup-files-list-rclone
 +
rclone copy -v  "etc/e-smith/templates-custom/" "smeserver-dropbox:backup/smeserver/etc/e-smith/templates-custom/"
 +
rclone copy -v  "etc/e-smith/templates-user-custom/" "smeserver-dropbox:backup/smeserver/etc/e-smith/templates-user-custom/"
 +
rclone copy -v  etc/ssh smeserver-dropbox:backup/smeserver/ssh
 +
rclone copy -v  root smeserver-dropbox:backup/smeserver/root
 +
rclone copy -v  home/e-smith/ smeserver-dropbox:backup/smeserver/home/ --exclude "tmp/**"
 +
</pre>
  
−
rclone mkdir smeserver-dropbox:backup/smeserver/home
+
=== Incremental Backup ===
  
−
rclone mkdir smeserver-dropbox:backup/smeserver/ssh
+
holk has documented a use of rclone that will give an incremental backup for up to 7 days (or whatever you might want)
  
−
rclone mkdir smeserver-dropbox:backup/smeserver/root
+
See here:
  
−
rclone copy -v  / smeserver-dropbox:backup/smeserver --files-from backup-files-list-rclone
+
https://forums.contribs.org/index.php/topic,53562.msg278216.html#msg278216
  
−
rclone copy -v  "etc/e-smith/templates-custom/" "smeserver-dropbox:backup/smeserver/etc/e-smith/templates-custom/"
+
=== Encrypted ===
  
−
rclone copy -v  "etc/e-smith/templates-user-custom/" "smeserver-dropbox:backup/smeserver/etc/e-smith/templates-user-custom/"
+
Rclone allows the remote data to be encrypted by specifying an intermediate remote which does the encrypting.
  
−
rclone copy -v  etc/ssh smeserver-dropbox:backup/smeserver/ssh
+
[https://rclone.org/crypt/ See here.]
  
−
rclone copy -v  root smeserver-dropbox:backup/smeserver/root
+
I have not tried this, but it looks pretty easy to add in and does not involve the SMEServer software.
  
−
rclone copy -v  home/e-smith/ smeserver-dropbox:backup/smeserver/home/ --exclude "tmp/**"
+
=== Restore ===
  
−
</code>
+
I assume that you can restore by just copying the data back from the cloud provider and then doing a signal-event post-upgrade; signal-event reboot;
  
 
=== Uninstall ===
 
=== Uninstall ===
 +
 +
Just delete /usr/bin/rclone and the config file /root/.config/rclone/rclone.conf
  
 
=== Bugs ===
 
=== Bugs ===
Line 148: Line 157:
  
 
<!-- list of category you want to see this page in -->
 
<!-- list of category you want to see this page in -->
−
[[Category: Contrib]]
+
[[Category: Howto]]
 +
[[Category: rclone]]
 +
[[Category: backup]]
 
<!-- Please keep there the template revision  number as is -->
 
<!-- Please keep there the template revision  number as is -->

Latest revision as of 06:11, 7 November 2021

Geographylogo.png Languages: 

English




rclone
NeedImage.svg
rclone logo
Maintainerbrianr
Urlhttps://wiki.contribs.org
LicenceGPL
Category

cli

Tags cli, centos, SME, install


Warning.png Warning:
Only for testing purpose, please report in bugzilla


Maintainer

Brian Read

Version

Requires command level knowledge to install and run.

Description

Rclone is a program written in Go to allow local files to be saved to one of many cloud servers. It tries to emulate rsync.

Current cloud providers supported are:


  • Alibaba Cloud (Aliyun) Object Storage System (OSS)
  • Amazon Drive (See note)
  • Amazon S3
  • Backblaze B2
  • Box
  • Ceph
  • DigitalOcean Spaces
  • Dreamhost
  • Dropbox
  • FTP
  • Google Cloud Storage
  • Google Drive
  • HTTP
  • Hubic
  • Jottacloud
  • IBM COS S3
  • Koofr
  • Memset Memstore
  • Mega
  • Microsoft Azure Blob Storage
  • Microsoft OneDrive
  • Minio
  • Nextcloud
  • OVH
  • OpenDrive
  • Openstack Swift
  • Oracle Cloud Storage
  • ownCloud
  • pCloud
  • put.io
  • QingStor
  • Rackspace Cloud Files
  • Scaleway
  • SFTP
  • Wasabi
  • WebDAV
  • Yandex Disk
  • The local filesystem

I have only tested it against Dropbox.

Installation

Rclone is made available as a statically linked executible and can be installed as follows: (https://rclone.org/install/) 

curl -O https://downloads.rclone.org/rclone-current-linux-amd64.zip
unzip rclone-current-linux-amd64.zip
cd rclone-*-linux-amd64
sudo cp rclone /usr/bin/
sudo chown root:root /usr/bin/rclone
sudo chmod 755 /usr/bin/rclone

Note that this loads the 64 bit version, other versions are available.

Then you can run the configuration process: 

rclone config

Choose your cloud provider and follow the instructions for a headless server.

For Dropbox I had to run rclone on a desktop with a browser, giving permission for rclone to be allowed to connect to Dropbox and then copy and paste a key string across to the smeserver.

Useage

Here is the shell script that I developed and tested to see if it would work: 

!/bin/sh
signal-event pre-backup
cd /
rclone mkdir smeserver-dropbox:backup
rclone mkdir smeserver-dropbox:backup/smeserver
rclone mkdir smeserver-dropbox:backup/smeserver/etc/e-smith/templates-user-custom
rclone mkdir smeserver-dropbox:backup/smeserver/etc/e-smith/templates-custom
rclone mkdir smeserver-dropbox:backup/smeserver/home
rclone mkdir smeserver-dropbox:backup/smeserver/ssh
rclone mkdir smeserver-dropbox:backup/smeserver/root
rclone copy -v  / smeserver-dropbox:backup/smeserver --files-from backup-files-list-rclone
rclone copy -v  "etc/e-smith/templates-custom/" "smeserver-dropbox:backup/smeserver/etc/e-smith/templates-custom/"
rclone copy -v  "etc/e-smith/templates-user-custom/" "smeserver-dropbox:backup/smeserver/etc/e-smith/templates-user-custom/"
rclone copy -v  etc/ssh smeserver-dropbox:backup/smeserver/ssh
rclone copy -v  root smeserver-dropbox:backup/smeserver/root
rclone copy -v  home/e-smith/ smeserver-dropbox:backup/smeserver/home/ --exclude "tmp/**"

Incremental Backup

holk has documented a use of rclone that will give an incremental backup for up to 7 days (or whatever you might want)

See here:

https://forums.contribs.org/index.php/topic,53562.msg278216.html#msg278216

Encrypted

Rclone allows the remote data to be encrypted by specifying an intermediate remote which does the encrypting.

See here.

I have not tried this, but it looks pretty easy to add in and does not involve the SMEServer software.

Restore

I assume that you can restore by just copying the data back from the cloud provider and then doing a signal-event post-upgrade; signal-event reboot;

Uninstall

Just delete /usr/bin/rclone and the config file /root/.config/rclone/rclone.conf

Bugs

Please raise bugs under the SME-Contribs section in bugzilla and select the smeserver-rclone component or use this link


Below is an overview of the current issues for this contrib:

No open bugs found.

Changelog

Only released version in smecontrib are listed here.

Retrieved from "https://wiki.koozali.org/index.php?title=Rclone&oldid=40593"