Comarison - of Huge volume with > 15.000.000 files does't finish and gets slower and slower

[DerGue]

I want to check if data-transfer from one volume to another was successful using FreeFileSync.

The volume has > 15.000.000 files.
I use 500 threads on source and 500 threads for destination.

The first 1.000.000 are compared blazing fast within 10 seconds.
The more files free file sync compares, the slower it gets and it seems to get slower and slower.

After 30 minutes and 16.000.000 files the counter increases very very slow.

Is this normal behaviour?

[xCSxXenon]

Limitation of memory?
https://freefilesync.org/faq.php#limitations

[TallBob]

My take on this is that is way too many threads. My rule of thumb for threading is 2 to 4 threads per physical disk. Here's my reasoning: A physical disk can only perform 1 read or write operation at a time. The Master File Table can only be updated one file at a time. You can read from one disk while writing to the other. This logic also applies to multi-volume storage spaces, RAIDs and SSD "disks".

Try 4 threads on the source and 4 on the target. Or even 2,2.