Only writing delta when synchronizing a file

[Tigrou]

Possible optimization: when synchronizing a file, only the delta (what has been modified) could be written to the file (instead of a normal copy of the file).

This would be useful for synchronizing large files (several GB) where only a few MB have been modified. This is the case with VM disk images, VeraCrypt encrypted containers and so on.

The advantage would be potentially faster synchronization and less wear and tear on the device. This wouldn't be useful in all cases (as you need additional read operations), but for devices with slow write operations (such as USB sticks), it would help. This should not be enabled by default, but only as an option (e.g. delta write).

The algorithm first checks that the files have the same size (if not, normal copy is used). It then compares fixed-size blocks (e.g. 32KB) and copies them (writes to destination file) if necessary.

[Tigrou]

Here is a python script example:
https://gist.github.com/tigrouind/46edacf9b53dd34af3b01f0a94120bd1

[Plerry]

The algorithm first checks that the files have the same size (if not, normal copy is used). It then compares fixed-size blocks (e.g. 32KB) and copies them (writes to destination file) if necessary.

It is very rare that files that have been modified and thus require syncing have the same size (as their previously synced version).
So, according to your algorithm, files that requiring syncing are almost always copied normally.
Therefore there is only a marginal benefit to a block wise compare and, when necessary, update, and thus little to gain.

[Tigrou]

It is very rare that files that have been modified and thus require syncing have the same size (as their previously synced version).
So, according to your algorithm, files that requiring syncing are almost always copied normally.
Therefore there is only a marginal benefit to a block wise compare and, when necessary, update, and thus little to gain.

It's the case of Virtual Machine images (eg: VDI, VMDK, VHD...), disk images (eg: IMG, ISO), VeraCrypt containers (.HC). Those are usually several GB and have a fixed size. I use FreeFileSync to sync between USB drives and portable disks and have few cases like that.