If I need to synchronize a unique non-directory object, say a file.
If I get it right, I have to add the including directory and
add an include filter for that; containing the filename.
Correct?
In my case there are directories with 10,000s of small files (process data),
where I need to synchronize only a few explicit ones, given by non-placeholder names.
Now when synchronizing:
Does FFS traverses the complete directory content and filter each of the
thousands of files for match of the explicit given names in the include list?
(This might take relative long I suspect?)
Synchronizing unique non-directory objects
- Posts: 36
- Joined: 30 Jun 2025
-
- Posts: 2946
- Joined: 22 Aug 2012
Correct.If I need to synchronize a unique non-directory object, say a file.
If I get it right, I have to add the including directory and
add an include filter for that; containing the filename.
Correct?
You sync the left and right base location(s), which must be folders/directories.
You can then limit the scope of your sync via the Include and/or Exclude filter.
In the Compare phase FreeFileSync (FFS) determines the proposed sync actions, based on the selected sync variant (Mirror, Update, Two-way or Custom), taking into account your Include and Exclude Filter rules.Now when synchronizing:
Does FFS traverses the complete directory content and filter each of the
thousands of files for match of the explicit given names in the include list?
(This might take relative long I suspect?)
In the FFS GUI, the user can then review and optionally manually alter the proposed sync actions.
During the Synchronization phase FFS simply executes the (potentially altered) proposed sync actions.
- Posts: 36
- Joined: 30 Jun 2025
Thank you for you reply.
However I know all that.
Actually you didn't get my point.
It's about implementation effiency when synchronizing.
(It's a question/ suggestion for the devs.)
When I need only one (or few) files from a source directoy that contains
ten thousands of files, my guess is that it is no good idea
to iterate the whole lot and filter all of them.
In the case of single file sync it's probably much faster
to execute only one copy command for that very file.
EDIT:
This doesn't hold only for directoys, but also for directorys
(and probably also for directories).
However I know all that.
Actually you didn't get my point.
It's about implementation effiency when synchronizing.
(It's a question/ suggestion for the devs.)
When I need only one (or few) files from a source directoy that contains
ten thousands of files, my guess is that it is no good idea
to iterate the whole lot and filter all of them.
In the case of single file sync it's probably much faster
to execute only one copy command for that very file.
EDIT:
This doesn't hold only for directoys, but also for directorys
(and probably also for directories).
- Posts: 1202
- Joined: 8 May 2006
> (This might take relative long I suspect?)
Did you test?
200K small files < 1000 bytes each, 1 directory tree, 100K in base directory, 100K in subdirectory
That took (literally) an instant to come up with a single wanted file.
(SSD & again 1 directory tree with only 1 subdirectory within, on a local, fast disk.)
(A more complex directory structure, going across a network, I'd expect ? substantially longer time, though still not anywhere near exorbitant - though one would have to test.)
Did you test?
200K small files < 1000 bytes each, 1 directory tree, 100K in base directory, 100K in subdirectory
That took (literally) an instant to come up with a single wanted file.
(SSD & again 1 directory tree with only 1 subdirectory within, on a local, fast disk.)
(A more complex directory structure, going across a network, I'd expect ? substantially longer time, though still not anywhere near exorbitant - though one would have to test.)
- Posts: 36
- Joined: 30 Jun 2025
Thank you for the test.
However actually that issue isn't due to user's experience, but algorithm-wise.
(As I wrote: for the devs).
I wrote: "... relative long...", with the focus on "relative".
Iterating and filtering a 200K list to identify 1 file where the name
is known already will take a multiple of time,
beside being algorithmic unnecessary.
Moreover I'm talking of a large data set to syncronize
(so neither source nor dest is SSD)
However actually that issue isn't due to user's experience, but algorithm-wise.
(As I wrote: for the devs).
I wrote: "... relative long...", with the focus on "relative".
Iterating and filtering a 200K list to identify 1 file where the name
is known already will take a multiple of time,
beside being algorithmic unnecessary.
Moreover I'm talking of a large data set to syncronize
(so neither source nor dest is SSD)
-
- Posts: 4866
- Joined: 11 Jun 2019
You worded your post horribly if you were attempting to provide feedback and direction for the dev. The original post includes two questions: one asking if your steps were correct and the other asking how the FFS code works. Those questions were answered correctly.
With that said, and as far as I know, FFS still has to compare every item to the filter settings to know if it needs to include it or exclude it, I don't know how this is done. The source code would answer definitively, as could Zenju, or testing could give a decent idea
With that said, and as far as I know, FFS still has to compare every item to the filter settings to know if it needs to include it or exclude it, I don't know how this is done. The source code would answer definitively, as could Zenju, or testing could give a decent idea