Difference between revisions of "Rclone"
(Created page with "{{Languages}} <!-- here we define the contrib name variable --> <!-- we get the page title, remove suffix for translated version; if needed you can define there with the value...") |
Unnilennium (talk | contribs) |
||
(29 intermediate revisions by 2 users not shown) | |||
Line 19: | Line 19: | ||
===Maintainer=== | ===Maintainer=== | ||
<!-- here you need to file your username and name --> | <!-- here you need to file your username and name --> | ||
− | [[User:brianr|Brian Read]] | + | [[User:brianr|Brian Read]] |
=== Version === | === Version === | ||
<!-- keep this first element as is, you can add some if needed --> | <!-- keep this first element as is, you can add some if needed --> | ||
− | {{#smeversion: {{#var:smecontribname}} }} | + | <!-- {{#smeversion: {{#var:smecontribname}} }}Not built into an rpm. --> |
+ | |||
+ | Requires command level knowledge to install and run. | ||
=== Description === | === Description === | ||
+ | [https://rclone.org/ Rclone] is a program written in Go to allow local files to be saved to one of many cloud servers. It tries to emulate rsync. | ||
+ | |||
+ | Current cloud providers supported are: | ||
+ | |||
+ | |||
+ | * Alibaba Cloud (Aliyun) Object Storage System (OSS) | ||
+ | * Amazon Drive (See note) | ||
+ | * Amazon S3 | ||
+ | * Backblaze B2 | ||
+ | * Box | ||
+ | * Ceph | ||
+ | * DigitalOcean Spaces | ||
+ | * Dreamhost | ||
+ | * Dropbox | ||
+ | * FTP | ||
+ | * Google Cloud Storage | ||
+ | * Google Drive | ||
+ | * HTTP | ||
+ | * Hubic | ||
+ | * Jottacloud | ||
+ | * IBM COS S3 | ||
+ | * Koofr | ||
+ | * Memset Memstore | ||
+ | * Mega | ||
+ | * Microsoft Azure Blob Storage | ||
+ | * Microsoft OneDrive | ||
+ | * Minio | ||
+ | * Nextcloud | ||
+ | * OVH | ||
+ | * OpenDrive | ||
+ | * Openstack Swift | ||
+ | * Oracle Cloud Storage | ||
+ | * ownCloud | ||
+ | * pCloud | ||
+ | * put.io | ||
+ | * QingStor | ||
+ | * Rackspace Cloud Files | ||
+ | * Scaleway | ||
+ | * SFTP | ||
+ | * Wasabi | ||
+ | * WebDAV | ||
+ | * Yandex Disk | ||
+ | * The local filesystem | ||
+ | I have only tested it against Dropbox. | ||
=== Installation === | === Installation === | ||
+ | Rclone is made available as a statically linked executible and can be installed as follows: (https://rclone.org/install/) | ||
+ | |||
+ | <pre> | ||
+ | curl -O https://downloads.rclone.org/rclone-current-linux-amd64.zip | ||
+ | unzip rclone-current-linux-amd64.zip | ||
+ | cd rclone-*-linux-amd64 | ||
+ | sudo cp rclone /usr/bin/ | ||
+ | sudo chown root:root /usr/bin/rclone | ||
+ | sudo chmod 755 /usr/bin/rclone | ||
+ | </pre> | ||
+ | |||
+ | Note that this loads the 64 bit version, other versions are available. | ||
+ | |||
+ | Then you can run the configuration process: | ||
+ | |||
+ | <pre>rclone config</pre> | ||
+ | |||
+ | Choose your cloud provider and follow the instructions for a headless server. | ||
+ | |||
+ | For Dropbox I had to run rclone on a desktop with a browser, giving permission for rclone to be allowed to connect to Dropbox and then copy and paste a key string across to the smeserver. | ||
+ | |||
+ | === Useage === | ||
+ | |||
+ | Here is the shell script that I developed and tested to see if it would work: | ||
+ | |||
+ | <pre> | ||
+ | !/bin/sh | ||
+ | signal-event pre-backup | ||
+ | cd / | ||
+ | rclone mkdir smeserver-dropbox:backup | ||
+ | rclone mkdir smeserver-dropbox:backup/smeserver | ||
+ | rclone mkdir smeserver-dropbox:backup/smeserver/etc/e-smith/templates-user-custom | ||
+ | rclone mkdir smeserver-dropbox:backup/smeserver/etc/e-smith/templates-custom | ||
+ | rclone mkdir smeserver-dropbox:backup/smeserver/home | ||
+ | rclone mkdir smeserver-dropbox:backup/smeserver/ssh | ||
+ | rclone mkdir smeserver-dropbox:backup/smeserver/root | ||
+ | rclone copy -v / smeserver-dropbox:backup/smeserver --files-from backup-files-list-rclone | ||
+ | rclone copy -v "etc/e-smith/templates-custom/" "smeserver-dropbox:backup/smeserver/etc/e-smith/templates-custom/" | ||
+ | rclone copy -v "etc/e-smith/templates-user-custom/" "smeserver-dropbox:backup/smeserver/etc/e-smith/templates-user-custom/" | ||
+ | rclone copy -v etc/ssh smeserver-dropbox:backup/smeserver/ssh | ||
+ | rclone copy -v root smeserver-dropbox:backup/smeserver/root | ||
+ | rclone copy -v home/e-smith/ smeserver-dropbox:backup/smeserver/home/ --exclude "tmp/**" | ||
+ | </pre> | ||
+ | |||
+ | === Incremental Backup === | ||
+ | |||
+ | holk has documented a use of rclone that will give an incremental backup for up to 7 days (or whatever you might want) | ||
+ | |||
+ | See here: | ||
+ | |||
+ | https://forums.contribs.org/index.php/topic,53562.msg278216.html#msg278216 | ||
+ | |||
+ | === Encrypted === | ||
+ | |||
+ | Rclone allows the remote data to be encrypted by specifying an intermediate remote which does the encrypting. | ||
+ | |||
+ | [https://rclone.org/crypt/ See here.] | ||
+ | |||
+ | I have not tried this, but it looks pretty easy to add in and does not involve the SMEServer software. | ||
+ | |||
+ | === Restore === | ||
+ | |||
+ | I assume that you can restore by just copying the data back from the cloud provider and then doing a signal-event post-upgrade; signal-event reboot; | ||
=== Uninstall === | === Uninstall === | ||
+ | |||
+ | Just delete /usr/bin/rclone and the config file /root/.config/rclone/rclone.conf | ||
=== Bugs === | === Bugs === | ||
Line 46: | Line 157: | ||
<!-- list of category you want to see this page in --> | <!-- list of category you want to see this page in --> | ||
− | [[Category: | + | [[Category: Howto]] |
+ | [[Category: rclone]] | ||
+ | [[Category: backup]] | ||
<!-- Please keep there the template revision number as is --> | <!-- Please keep there the template revision number as is --> |
Latest revision as of 06:11, 7 November 2021
rclone logo | |
Maintainer | brianr |
---|---|
Url | https://wiki.contribs.org |
Licence | GPL |
Category | |
Tags | cli, centos, SME, install |
Maintainer
Version
Requires command level knowledge to install and run.
Description
Rclone is a program written in Go to allow local files to be saved to one of many cloud servers. It tries to emulate rsync.
Current cloud providers supported are:
- Alibaba Cloud (Aliyun) Object Storage System (OSS)
- Amazon Drive (See note)
- Amazon S3
- Backblaze B2
- Box
- Ceph
- DigitalOcean Spaces
- Dreamhost
- Dropbox
- FTP
- Google Cloud Storage
- Google Drive
- HTTP
- Hubic
- Jottacloud
- IBM COS S3
- Koofr
- Memset Memstore
- Mega
- Microsoft Azure Blob Storage
- Microsoft OneDrive
- Minio
- Nextcloud
- OVH
- OpenDrive
- Openstack Swift
- Oracle Cloud Storage
- ownCloud
- pCloud
- put.io
- QingStor
- Rackspace Cloud Files
- Scaleway
- SFTP
- Wasabi
- WebDAV
- Yandex Disk
- The local filesystem
I have only tested it against Dropbox.
Installation
Rclone is made available as a statically linked executible and can be installed as follows: (https://rclone.org/install/)
curl -O https://downloads.rclone.org/rclone-current-linux-amd64.zip unzip rclone-current-linux-amd64.zip cd rclone-*-linux-amd64 sudo cp rclone /usr/bin/ sudo chown root:root /usr/bin/rclone sudo chmod 755 /usr/bin/rclone
Note that this loads the 64 bit version, other versions are available.
Then you can run the configuration process:
rclone config
Choose your cloud provider and follow the instructions for a headless server.
For Dropbox I had to run rclone on a desktop with a browser, giving permission for rclone to be allowed to connect to Dropbox and then copy and paste a key string across to the smeserver.
Useage
Here is the shell script that I developed and tested to see if it would work:
!/bin/sh signal-event pre-backup cd / rclone mkdir smeserver-dropbox:backup rclone mkdir smeserver-dropbox:backup/smeserver rclone mkdir smeserver-dropbox:backup/smeserver/etc/e-smith/templates-user-custom rclone mkdir smeserver-dropbox:backup/smeserver/etc/e-smith/templates-custom rclone mkdir smeserver-dropbox:backup/smeserver/home rclone mkdir smeserver-dropbox:backup/smeserver/ssh rclone mkdir smeserver-dropbox:backup/smeserver/root rclone copy -v / smeserver-dropbox:backup/smeserver --files-from backup-files-list-rclone rclone copy -v "etc/e-smith/templates-custom/" "smeserver-dropbox:backup/smeserver/etc/e-smith/templates-custom/" rclone copy -v "etc/e-smith/templates-user-custom/" "smeserver-dropbox:backup/smeserver/etc/e-smith/templates-user-custom/" rclone copy -v etc/ssh smeserver-dropbox:backup/smeserver/ssh rclone copy -v root smeserver-dropbox:backup/smeserver/root rclone copy -v home/e-smith/ smeserver-dropbox:backup/smeserver/home/ --exclude "tmp/**"
Incremental Backup
holk has documented a use of rclone that will give an incremental backup for up to 7 days (or whatever you might want)
See here:
https://forums.contribs.org/index.php/topic,53562.msg278216.html#msg278216
Encrypted
Rclone allows the remote data to be encrypted by specifying an intermediate remote which does the encrypting.
I have not tried this, but it looks pretty easy to add in and does not involve the SMEServer software.
Restore
I assume that you can restore by just copying the data back from the cloud provider and then doing a signal-event post-upgrade; signal-event reboot;
Uninstall
Just delete /usr/bin/rclone and the config file /root/.config/rclone/rclone.conf
Bugs
Please raise bugs under the SME-Contribs section in bugzilla and select the smeserver-rclone component or use this link
Below is an overview of the current issues for this contrib:
Changelog
Only released version in smecontrib are listed here.