Sync subset of folders

Discuss new features and functions
wing
Posts: 7
Joined: 6 Jun 2019

Post by wing • 06 Jun 2019, 10:15

Hi, I'm new here. I am looking for a way to sync a subset of folders. I am using to collaborate with other people but I don't need the entire contents of a folder. Let me explain.

Left - /Local/
Right - /Remote/

Left
/Local/2/
/Local/3/
/Local/9/

Right
/Remote/1/
/Remote/2/
/Remote/3/
/Remote/4/
/Remote/5/
/Remote/6/
/Remote/7/
/Remote/8/

What I need is folders 2, 3 and 9 gets synchronized. Not interested in folders 1,4,5,6,7,8.

So I want to create folder 9 on the right and then sync folders 2, 3 and 9. Essentially, I am only looking to sync a subset of the remote folder on the right but also if a folder does not exist on the right also create and sync that.

The only way I can do that right now is to manually add each subfolder but of course this does not scale very well. It is actually in the thousands of folders, too inefficient to manually add each time. Hope this helps understand my problem.

User avatar
Plerry
Posts: 707
Joined: 22 Aug 2012

Post by Plerry • 07 Jun 2019, 06:47

I'm not sure I fully get your use case, but I'll give it a shot.

Sticking to
Left - /Local/
Right - /Remote/
seems OK

and then you can use the Include filter (the funnel-icon at the top between the left and right separator) to include just
/2/*
/3/*
/9/*
instead of everything (*).
You can edit the Include (and Exclude) filter manually,
but you can also run a Compare and then right-click on one or more folders and add those to your filter definitions via the GUI.

Conversely, in some cases it is easier to include everything (*), and only exclude specific items.
Note that filter definitions have a (limited) wildcard capability, that may be useful for you.

wing
Posts: 7
Joined: 6 Jun 2019

Post by wing • 07 Jun 2019, 08:12

Plerry wrote:
07 Jun 2019, 06:47
and then you can use the Include filter (the funnel-icon at the top between the left and right separator) to include just
/2/*
/3/*
/9/*
Sorry if I didn't explain myself clearly.

That is exactly what I am trying to avoid. I would like to avoid manually adding folder pairs or filters.

Think of it as 100s of folders that I would like to work on locally. I can create new folders and then it is uploaded (sync using freefilesync) to the server. But I do not want the entire server worth of folders.

I.e.: Left (local) must be a subset of right (remote server). Sub-folder of the folder pair that exists on the right but does not exists on the left is ignored.

I think what is required is somehow, is an iteration to traverse through all sub-folders of the folder pair level on the left and then compare with sub-folders of the folder pair level on the right.

Code: Select all

for each sub-folder of the folder pair on the left {
  if sub-folder of the folder pair on the right exists, then sync
  if sub-folder of the folder pair on the right does not exist, then {
     create 
     sync 
   }
}

wing
Posts: 7
Joined: 6 Jun 2019

Post by wing • 07 Jun 2019, 08:35

PS: just to confuse everyone, the above pseudo code does not deal with deletion of sub-folders of the folder pair and that is good. If I delete a sub-folder of the folder pair on the left, I do not want the deletion to propagate to the right.

Unless of course it is a deletion of the contents or the sub-folder of the sub-folder of the folder pair on the left then it is sync as per normal (which the above pseudo code does deal with).

User avatar
Plerry
Posts: 707
Joined: 22 Aug 2012

Post by Plerry • 07 Jun 2019, 10:29

Based on what you describe, it seems all changes will originate locally.
Then it seems a simple
Left - /Local/
Right - /Remote/
will do,
but you need to select the (left-to-right) Update variant for your Sync-settings (F8), so not the Two-way or Mirror variant.
This will sync your left location to the relevant parts in your right location, but will leave everything else in the right location intact.
Using the FFS Update variant will:
1) If they do not pre-exist, create files in the corresponding right location as far as they exist in the left to location.
2) If they do pre-exist, update files in the corresponding right location to match changes made to the left location
3) Leave files/folders pre-existing in the right location intact if they get deleted from the left location or do not exist in the left location.
Because of rule 3), all other folders that only exist in /Remote/ will not be affected.
However, files and folders deleted from the left side will not be deleted from the right side either if those would pre-exist there. But that seems to be exactly what you want.

wing
Posts: 7
Joined: 6 Jun 2019

Post by wing • 13 Jun 2019, 02:33

Plerry wrote:
07 Jun 2019, 10:29
Based on what you describe, it seems all changes will originate locally.
Then it seems a simple
Left - /Local/
Right - /Remote/
will do,
but you need to select the (left-to-right) Update variant for your Sync-settings (F8), so not the Two-way or Mirror variant.
This will sync your left location to the relevant parts in your right location, but will leave everything else in the right location intact.
Using the FFS Update variant will:
1) If they do not pre-exist, create files in the corresponding right location as far as they exist in the left to location.
2) If they do pre-exist, update files in the corresponding right location to match changes made to the left location
3) Leave files/folders pre-existing in the right location intact if they get deleted from the left location or do not exist in the left location.
Because of rule 3), all other folders that only exist in /Remote/ will not be affected.
However, files and folders deleted from the left side will not be deleted from the right side either if those would pre-exist there. But that seems to be exactly what you want.
Thanks for your reply. Pretty close but not quite. I guess it is probably best explained in a real world situation.

1. I am making a copy of all paperwork in folder A from the filing cabinet. I am going away from office.
2. The original paperwork folder A is left in filing cabinet in the office.
3a. I make changes to the copy of folder A away from office.
3b. Meanwhile, co-worker make changes to the original folder A in the office.
4. I come back to office and amend all changes (from myself and co-worker) to both my copy and the original folder A. (This is a Sync operation, not Update)

(5). If I no longer in charge of folder A, then digitally, I delete the my personal copy. Original folder is never ever destroyed (this is a big no-no).

The only way I can see this happening is to create a folder pair for each folder I am taking with me, but I am trying to avoid this as it is very numerous and changes very quickly.

wing
Posts: 7
Joined: 6 Jun 2019

Post by wing • 13 Jun 2019, 02:37

(5b) Also, I am allowed to create folder B while away from Office. A copy of folder B is created and then deposited to the filing cabinet when I am back at the office.

I hope this explains why the sync happens only greater or equal 1 folder level deep. I am not interested in taking the whole filing cabinet with me. The storage space required is too big. The closest thing in windows 10 would be offline files but I haven't found it to be very reliable and it is disabled by group policy by the system administrator.

BONUS (Also not sure if this is technically possible, as I don't want orphaned folders):
(6) I noticed folder A is not supposed to be called folder A, I renamed my copy of folder A to folder C while away from office. I come back to the office and also rename folder A in the filing cabinet to folder C.

User avatar
Plerry
Posts: 707
Joined: 22 Aug 2012

Post by Plerry • 14 Jun 2019, 08:50

Pfft! You have a very specific use case that, as far as I see it now, is presently not supported by FFS.
But I may be overlooking possibilities …

User avatar
Plerry
Posts: 707
Joined: 22 Aug 2012

Post by Plerry • 14 Jun 2019, 11:49

On second thought:
You may look into defining a Custom sync variant.
In the Synchronization settings (F8), you can select the Custom variant and can define your custom actions.
If we assume that you define your local location (e.g. C:/Local/), containing a subset of data, as your left location, and your shared remote location (e.g. D:/Remote/), containing all data, as your right location, then it seems defining your Custom sync settings as (from left icon to right icon):
  • Left; Category: Item exists on left side only; Action: Copy new item to the right
  • Mid-left; Category: Item left side is newer; Action: Update right item
  • Mid; Category: Conflict; Action: Leave as unresolved conflict (or some other choice, if applicable)
  • Mid-right; Category: Right side is newer; Action: Update left item
  • Right; Category: Item exists on right side only; Action: Do nothing
should do the trick for all but item 6.
You can change the action per Category by clicking on the corresponding Action icon.
You then cycle through the available options.

Unfortunately, you can not get your bonus, item 6, as "Detect moved files" seems to be unavailable for the Custom sync variant.

wing
Posts: 7
Joined: 6 Jun 2019

Post by wing • 18 Jun 2019, 09:32

Plerry wrote:
14 Jun 2019, 08:50
Pfft! You have a very specific use case that, as far as I see it now, is presently not supported by FFS.
But I may be overlooking possibilities …
Sorry about that.
Plerry wrote:
14 Jun 2019, 11:49
On second thought:
You may look into defining a Custom sync variant.
In the Synchronization settings (F8), you can select the Custom variant and can define your custom actions.
If we assume that you define your local location (e.g. C:/Local/), containing a subset of data, as your left location, and your shared remote location (e.g. D:/Remote/), containing all data, as your right location, then it seems defining your Custom sync settings as (from left icon to right icon):
  • Left; Category: Item exists on left side only; Action: Copy new item to the right
  • Mid-left; Category: Item left side is newer; Action: Update right item
  • Mid; Category: Conflict; Action: Leave as unresolved conflict (or some other choice, if applicable)
  • Mid-right; Category: Right side is newer; Action: Update left item
  • Right; Category: Item exists on right side only; Action: Do nothing
should do the trick for all but item 6.
You can change the action per Category by clicking on the corresponding Action icon.
You then cycle through the available options.

Unfortunately, you can not get your bonus, item 6, as "Detect moved files" seems to be unavailable for the Custom sync variant.
Yes. I have actually thought of that independently. I have tried that. Unfortunately, new files within subfolder have been inserted by my co-worker that I did not know about. This is where:
[*]Right; Category: Item exists on right side only; Action: Do nothing
is not sufficient. This is because I did not check for new created items, so I have made bad decisions out in the office without adequate information so this does not work for me in the long term.

I have created a freefilesync batch file and it looks like xml. It might be something that I can write code and automate in the future but for now this custom configuration will do for now.

wing
Posts: 7
Joined: 6 Jun 2019

Post by wing • 18 Jun 2019, 11:58

I realize now what I really need is a command line sync so that so I can create folder pairs on the fly.

Update: I have create a test command with 3 left and 3 right folders. The GUI pops up, which was not what I expected. It does allow me to manually and visually compare, which is a step I rather skip, but anyway I can live with that. It is better than manually creating folder pairs. The synchronisation method is mirror by default. I am guessing it is perhaps due to lack of synchronisation database files. I changed the method to two-way manually and then run the test. It prompts that the folders are missing. This was intentional. I want to test if it can create folders. Excellent. I closed freefilesync GUI and then check each folder. Folders were created as expected. ffs_db files was created for each folder. So far so good. I re-run the command. The GUI comes up with mirror again. Just a minor annoyance as it is only a couple of clicks but I think this is my solution. Next to generate the command with a for loop. I think I have solved my own problem. I don't think this will solve my rename at the 1st subfolder level problem but that was a bonus. Thanks Plerry for your replies.