Linux/Basic commands/rsync

rsync[1] is an advanced copy and synchronization tool including progress indicator.

Configuration files

edit
  • /etc/rsyncd.conf which includes don't compress options for some file extensions

Options

edit
-W, --whole-file
-z, --compress compresses the file data as it is sent to the destination machine using zlib compression library

Basic commands

edit
  • Copy files from remote server to local machine: rsync -chavzP --stats user@remote.host:/path/to/copy /path/to/local/storage[2] (explainshell[3])
-c, --checksum option can slow transfers specially for large files
-h, --human-readable
-a, --archive. Equivalent to -rlptgoD
-v, --verbose
-z, --compress
-P, --partial --progress. --partial keep partially transferred files
--stats, print statistics
-a, --archive. Equivalent to -rlptgoD
-r --recursive recurse into directoriees
-l --links copy symlinks as symlinks
-p --perms preserve permissions
-t --times preserve modification times
-g --group preserve group
-o --owner preserve owner (super-user only)
-D same as --devices --specials


rsync -chavzP --stats user@remote.host:/path/to/copy /path/to/local/storage
receiving incremental file list
.../...


  • rsync -a -v --ignore-existing src dst

Compression

edit

rsync support compression, but you can also compress your data with other external compression tools before using rsync. Such as bzip, gzip or xz


Compressing using xz[4]

  • xz -v -T0 file_to_compress
-v provides progress bar
-T0 use all available Threads, multi-threaded compression supported since 2014, version 5.2.0. [5]

Activities

edit
  1. Create a remote copy of your files for backup proposes using rsync
  2. Read StackOverflow questions about rsync: https://stackoverflow.com/questions/tagged/rsync?tab=Votes
  3. Understand the differences between cp, scp and rsync: https://stackoverflow.com/questions/20244585/how-does-scp-differ-from-rsync
  4. Learn Effect of Trailing Slash / in rsync: https://www.alibabacloud.com/blog/speeding-up-network-file-transfers-with-rsync_594337
  5. Learn use of --inplace for transferring large files and its important implications: http://man7.org/linux/man-pages/man1/rsync.1.html, also evaluate -c, --checksum impact on transfering large files
  6. Learn about -W, --whole-file copy files whole (without delta-xfer algorithm), implications and usage cases such as not making a diff copy.
  7. Learn about --sparse option and conflicts with --inplace

See also

edit
  1. http://man7.org/linux/man-pages/man1/rsync.1.html
  2. https://stackoverflow.com/questions/9090817/copying-files-using-rsync-from-remote-server-to-local-machine
  3. https://explainshell.com/explain?cmd=rsync+-chavzP+--stats+user%40remote.host%3A%2Fpath%2Fto%2Fcopy+%2Fpath%2Fto%2Flocal%2Fstorage
  4. https://linux.die.net/man/1/xz
  5. https://git.tukaani.org/?p=xz.git;a=blob;f=NEWS;hb=HEAD