Concurrent Sync
A concurrent sync tool which works similar to rsync. It supports syncing given sources with multiple targets
concurrently.
Requirements
Python >= 3.8 is required. (CPython and PyPy are both supported)
Installation
Concurrent Sync can be either installed directly via pip:
pip install concurrent-sync
Or it can be installed from the source:
git clone https://github.com/simsekhalit/concurrent-sync.git
python3 -m pip install ./concurrent-sync
Manual
$ python3 -m csync --help
usage: csync [-h] [--max-memory MAX_MEMORY] [--target TARGET] SOURCE [SOURCE ...] TARGET
A concurrent sync tool which works similar to rsync. It supports syncing given sources with multiple targets concurrently.
positional arguments:
SOURCE specify source directories/files
TARGET specify target directory
optional arguments:
-h, --help show this help message and exit
--max-memory MAX_MEMORY
specify allowed max memory usage as percent
--target TARGET specify additional target directories
For more information: https://github.com/simsekhalit/concurrent-sync
Concurrent Sync takes one or more source paths and one or more target paths as arguments.
All the given source paths are synced with each given target path concurrently.
- Only the missing or changed files are copied from source to target.
- While checking if a file is changed, its modification time and size are used similar to
rsync. - Trailing slash at the end of the source is interpreted in a similar way to
rsync.
Following Examples section clarifies the working mechanics in a more clear way.
Examples
1. One source path and one target path are given
Source
/mnt/Source
/mnt/Source/File1
/mnt/Source/File2
/mnt/Source/Folder1
/mnt/Source/Folder2
/mnt/Source/Folder2/File3
Target
/mnt/Target
Following command is executed:
python3 -m csync /mnt/Source /mnt/Target
After the sync is completed, target becomes:
/mnt/Target
/mnt/Target/Source
/mnt/Target/Source/File1
/mnt/Target/Source/File2
/mnt/Target/Source/Folder1
/mnt/Target/Source/Folder2
/mnt/Target/Source/Folder2/File3
2. Two source paths and one target are given
Source 1
/mnt/Source1
/mnt/Source1/File1
/mnt/Source1/File2
Source 2
/mnt/Source2
/mnt/Source2/File3
/mnt/Source2/File4
Target
/mnt/Target
Following command is executed:
python3 -m csync /mnt/Source1 /mnt/Source2 /mnt/Target
After the sync is completed, target becomes:
/mnt/Target
/mnt/Target/Source1
/mnt/Target/Source1/File1
/mnt/Target/Source1/File2
/mnt/Target/Source2
/mnt/Target/Source2/File3
/mnt/Target/Source2/File4
3. Source with trailing slash and target are given
Source
/mnt/Source
/mnt/Source/File1
/mnt/Source/File2
Target
/mnt/Target
Following command is executed:
python3 -m csync /mnt/Source/ /mnt/Target
After the sync is completed, target becomes:
/mnt/Target
/mnt/Target/File1
/mnt/Target/File2
4. Source and target with common paths
While syncing subdirectories of source paths with target paths, redundant files/folders are removed.
Source
/mnt/Source
/mnt/Source/Folder
/mnt/Source/Folder/File1
/mnt/Source/Folder/File2
Target
/mnt/Target
/mnt/Target/Source
/mnt/Target/Source/Folder
/mnt/Target/Source/Folder/File3
Following command is executed:
python3 -m csync /mnt/Source /mnt/Target
After the sync is completed, target becomes:
/mnt/Target
/mnt/Target/Source
/mnt/Target/Source/Folder
/mnt/Target/Source/Folder/File1
/mnt/Target/Source/Folder/File2
Since File3 is no longer in the source path it’s deleted from the target as well.
5. One source path and two target paths are given
Source
/mnt/Source
/mnt/Source/File1
/mnt/Source/File2
Target 1
/mnt/Target1
Target 2
/mnt/Target2
Following command is executed:
python3 -m csync /mnt/Source /mnt/Target1 --target /mnt/Target2
After the sync is completed, targets become:
Target 1
/mnt/Target1
/mnt/Target1/Source
/mnt/Target1/Source/File1
/mnt/Target1/Source/File2
Target 2
/mnt/Target2
/mnt/Target2/Source
/mnt/Target2/Source/File1
/mnt/Target2/Source/File2
