Sync more than two sites at same time

[jmeca]

Hi, is posibol to sync three sites or more at same time.

I think would be nice, and save time for enterprises

Thank you very much for your help

[xCSxXenon]

Yes

[Plerry]

You can define multiple left-right folder pairs, if desired with different sync and/or filter settings.

The donation edition supports parallel file copy. See the manual section on Performance.

[Dj3ross]

Hi,
As I have a related problem, I think it's best to use this thread instead of creating a new one.

I tried syncing multiple sources / destinations at the same time, and found out there is no parallel copy.
Let me explain.

I want to sync 2 sources, to 3 destinations each, like this :
Disk1/Folder1 -> DiskA/Dest1
Disk1/Folder1 -> DiskB/Dest2
Disk1/Folder1 -> DiskC/Dest3
Disk2/Folder2 -> DiskA/Dest4
Disk2/Folder2 -> DiskB/Dest5
Disk2/Folder2 -> DiskC/Dest6

However, whatever number of threads I put in the source / destination Disks, FFF only processes these pairs sequentially, not in parallel.
It seems it copies all data relative to DiskA first without concurrent reads from different sources, then all data relative to DiskB, then DiskC, just like it is grouped on the left overview pane.

Instead, what I'm looking for is for it to processes at least the first set of source data (Disk1/Folder1) to all 3 destinations at once, then Disk2/Folder2.
Or like it is doing right now, but with concurrent access from different sources when there are more than one.
Or everything at once if it's not possible to do otherwise.
The reasoning behind that : the source drives are local SSDs, so they can handle the multiple reads at once for the destination drives which are 1 local backup HDD and 2 network drives.

Can you tell me if there is an issue on my end ?
Or is it working as intended, and if that's the case, is any real parallel copy scheduled ?

Thanks for your help.
Regards,
Dje

[Plerry]

If you add all left-right pairs to one-and-the-same sync, there is no parallel file-copy, except when using the donation edition.
However, you may try having multiple instances of FFS running your different syncs.
If you use directory locking (=default), you can only run two syncs; one for the first three left-right pairs and a second one for the last three left-right pairs.
If you disable directory locking (safe if these are one-way syncs, and no other syncs make changes to the left location(s), you could run all six in different FFS instances.

[xCSxXenon]

Unfortunately, FFS doesn't behave that way. It would actually make some syncs slower on HDDs, since the destinations wouldn't be completely in sync, the source would be reading from different locations. Parallel threads is for network transfers that limit bandwidth per connection, by opening multiple connections. You may think it would be faster to do what you are describing, but probably not in real world use.

[Dj3ross]

Hi,
Thank you both for your answer.

@Pierry : I didn't mention it explicitly, but I use the donation edition of course, otherwise I wouldn't be able to modify the number of threads. But I should have said so, my mistake.
Thanks for the tip, I'll check if this is a viable option, as locking shouldn't be an issue (mirror sync).
Would you know a way to open multiple instances on macOS, apart from duplicating the app ? -> Nevermind, found it with a quick research.

@xCSxXenon : I get that it would probably be slower for HDD, completely understandable. Hence it should be optional, just like multiple threads.
But today SSD are a common thing, and they can give a serious boost to some specific workloads like mine.
Maybe I'll have to re-think the way I do these sync then, or change tool — I'd rather not, I really like FFS.

If FFS developer have some comment about this, I'd be glad to read his thoughts.

Anyway, thanks again.
Regards.

[Zenju]

FFF only processes these pairs sequentially, not in parallel.

Yes, this is the current behavior. But it's not optimal. Ideally FFS would run multiple folder pairs at the same time respecting the limits set in "parallel file operations". So it's a task scheduling problem (but not entirely trivial: binary comparison already behaves this way, but the algorithm is rather complicated) => ToDo

[Dj3ross]

Thanks Zenju.
At least I know it will come one day, that's nice.
For now I tried the workaround @Pierry gave me : what a massive difference !
But a hell to manage when using manually, creating separate profiles, multiple instance, etc...
I'll be glad when it will be native :)

Thank you all.
Regards.

[Dj3ross]

=> ToDo

Hi Zenju !
Just checking, is it still on your ToDo list ?
Do we have a chance to see it anytime soon ?

Thanks 😉
Regards.