Suggestion: All sync.ffs_db files can be put in 1 db file in the FreeFileSync folder

[travelid]

Hi,
Thank you for creating an incredible piece of software!
It is by far the best I've ever found (and I've found many many :) during my quest for the ultimate and the fastest worry-free freeware backup software. It is unbelievably fast!

Here's a couple of suggestions to make it even greater!

1) It would be great if FreeFileSync puts all database files in its own folder, so there's not a sync.ffs_db file in every parent folder synced. Like many advanced users I have my "Show hidden files" enabled on my WinXP, so I see the sync.ffs_db files in many places, on my Desktop included, and it's a big hard to get around them when I am often selecting all files in a folder to move/delete/batch-rename them. No other software that I've tried leaves db files in other folders.

2) Would be great if there was a built-in option to monitor for changes real-time or at least every 5 minutes and auto-backup all changes to files. Extremely useful when working on those big excel sheets and writing in doc files, as the user never has to worry about closing their work files to manually back them up with FreeFileSync on a USB key or external HDD.

Hope this help!
Again, I thank you from my heart for your wonderful work!

[Pako]

SyncToy leaves db files all over the place. I like the sync.ffs_db files because I am running script to watch the central sync location for changed *.ffs_db files and to report last sync times. It would be great if I knew way to query this db files to see what is inside and if it can be useful for me. For example, I want to know who was the last user that synced files and what files they did sync. Currently I accomplish this by analyzing the backed up files and the creation/deletion of _lock file.

Regarding your second point, check the RealTimeSync included with FFS. I think that's what you are looking for. Consult the help file on usage, otherwise you may find yourself frustrated.

[greydawn]

I would like to resurrect and +1 the first item. Can an expert option be included to have all sync.ffs_db files placed into FFS' ProgramData directory rather than at the root of the location being synced? As a Sys Admin the file rename/move detection is a critical piece of why we are hoping to use FFS but unfortunately the fact that these files are being placed in user accessible directories is causing some confusion and other problems due to the way our network file systems are structured.

A slightly less appealing solution would be to mark these files as System,Hidden instead of just Hidden... our users have Hidden file display turned on by default but marking the ffs_db files as System would still prevent them from being seen by most end users.

[wm-sf]

is it the fact the db files can be seen or that they might be deleted or similar that is the problem with them being where they are at present?

[greydawn]

There are 4 issues in our scenario:

1. The current location of the sync.ffs_db files is a user readable location under a set of managed folders. While they can't modify/delete the files/folders here the presence of unexpected file data may certainly result in unwanted calls and inquiries to our help desk.

2. The current location that FFS wants to write these DB files to is a DFS namespace. This presents challenges since the folders at this level are managed by DFS and do not tolerate very well the presence of "foreign" files. One example is that removing a DFS folder from the namespace should normally result in the actual folder being deleted from the file system but this cannot be accomplished with foreign content in the folder resulting in an inconsistency between the DFS folder structure and the namespace settings stored in AD. But perhaps most importantly, in some scenarios FFS may resolve the DFS root to a different namespace server than it has previously used. FFS then no longer has access to the latest sync.ffs_db and when it writes a new one those changes may also be orphaned.

3. We have a periodic script we run to look for DFS folder structure that is out of sync with the namespace as stored in AD and this script removes any unknown files/folders. This results in the deletion of the sync.ffs_db (since it is not defined in DFS).

4. The account that runs FFS in batch mode is a highly privileged account that bypasses security controls. Hypothetically, in our scenario any end user could figure out how to read these sync.ffs_db files and thereby access privileged information (probably limited to file names, paths, etc.) There are scenarios under some of our contracts where even the names of files must remain secure and limited to specific individuals with clearance to that information. There are permissions I can implement in our DFS to avoid this but it would be easier for me if the sync.ffs_db were written to a central location on a non- user-accessible server that I can better control... especially since 1-3 are still problems.

[Zenju]

> in some scenarios FFS may resolve the DFS root to a different namespace server

Are you sure that "two way" is the variant you need? If the data is available multiple times, then it's logically correct that FFS creates multiple database files, one for each physical incarnation of a data set. But maybe in your scenario the "mirror" variant is a better fit, in which case no database is needed.

Anyway, as a workaround it's always possible to set up some arbitrary pair of folders and "mount in" the real folders for synchronization with symlinks. This way you have control on where the database files are stored.

[greydawn]

> in some scenarios FFS may resolve the DFS root to a different namespace server

Are you sure that "two way" is the variant you need? If the data is available multiple times, then it's logically correct that FFS creates multiple database files, one for each physical incarnation of a data set. But maybe in your scenario the "mirror" variant is a better fit, in which case no database is needed.

Anyway, as a workaround it's always possible to set up some arbitrary pair of folders and "mount in" the real folders for synchronization with symlinks. This way you have control on where the database files are stored.

> Are you sure that "two way" is the variant you need?

We don't do two way. The FFS batch jobs are set up as a mirror with the option "Detect moved files" selected. This does create a sync.ffs_db in the source folder.

> If the data is available multiple times, then it's logically correct that FFS creates multiple database files

The data itself is not available multiple times. DFS allows for multiple namespace servers (for redundancy) that each host the upper levels of a directory tree (no files). As you browse into the tree you encounter leaf nodes that transparently redirect the Windows client out to a specific file server / share. The effect is that are browsing a single file system but in reality you are being redirected to content that may be on many individual servers. However, generally the content only exists once, on one single server, so it doesn't matter which namespace server redirected you to that content. The namespace servers themselves have the upper level folder structure automatically synced by the DFS service.

The issue here is that since I am backing up a DFS path (and all of the content that may be on multiple servers below that point) FFS is writing its sync.ffs_db folder into the DFS namespace folders... in a specific namespace server (remember, all namespace servers look identical up to this point) However, DFS doesn't recognize file content at this location and doesn't automatically sync that file to all other DFS namespace servers the way it keeps the folder structure in sync. If the next time Windows resolves my source path \\<Domain>\Content\FolderToBackup using a different namespace server then to FFS it looks like the sync.ffs_db file is missing.

> Anyway, as a workaround it's always possible to set up some arbitrary pair of folders and "mount in" the real folders for synchronization with symlinks.

In this case that is not possible. The psuedo-folder structure that hosts the DFS namespace browsable structure on each DFS namespace server (and where FFS is writing its DB files) cannot contain symlinks... nor can these be creating at another location on the same disk to point back to the DFS namespace folders since the DFS Client cannot then properly resolve the DFS Links.

---

That's really just for your edification. In reality I think there may be various situations where it is infeasible or otherwise desirable to have the DB files placed somewhere other than in the source folder. I'm not even necessarily asking for this to be worked up in the GUI... just make this an advanced parameter in the .ffs_batch file and let me provide a complete path to read/write the sync.ffs_db for each folder pair (or the entire job as you see fit).

[wm-sf]

Aaron, it is obviously not my call but I think there are enough specifics about your situation that you might need (yet) another thread.

I'm thinking the combination of factors in your case is getting unusual.

As a casual reader I think you want mirror as the main FFS mode but also need moved files detection. In your case it is looking like these two options aren't going to work together. IIRC the use of moved files detection (resulting in the db files on both sides) was something that came up in conversation here.

Q: if you had a better way (independent of FFS) of dealing with moved stuff wouldn't you be able to turn that bit of FFS off and your DFS problem would be solved?

i.e. isn't the "home" issue why DFS isn't tracking the moves rather than why FFS can't track them the way you'd ideally like?

Stuff to think about.

[greydawn]

Aaron, it is obviously not my call but I think there are enough specifics about your situation that you might need (yet) another thread.

I'm thinking the combination of factors in your case is getting unusual.

As a casual reader I think you want mirror as the main FFS mode but also need moved files detection. In your case it is looking like these two options aren't going to work together. IIRC the use of moved files detection (resulting in the db files on both sides) was something that came up in conversation here.

Q: if you had a better way (independent of FFS) of dealing with moved stuff wouldn't you be able to turn that bit of FFS off and your DFS problem would be solved?

i.e. isn't the "home" issue why DFS isn't tracking the moves rather than why FFS can't track them the way you'd ideally like?

Stuff to think about.

> As a casual reader I think you want mirror as the main FFS mode but also need moved files detection. In your case it is looking like these two options aren't going to work together.

Mirror and moved files detection work just fine together. The issue isn't the functionality of FFS, its where the sync.ffs_db files are being created.

> Q: if you had a better way (independent of FFS) of dealing with moved stuff wouldn't you be able to turn that bit of FFS off and your DFS problem would be solved?

My problem isn't really anything to do with DFS, I'm just trying to explain why it is problematic that I can't control where the sync.ffs_db files are being stored. You are correct that if sync.ffs_db files don't show up where end users can see them that I don't have a problem.

> i.e. isn't the "home" issue why DFS isn't tracking the moves rather than why FFS can't track them the way you'd ideally like?

Something may be lost in translation here... I'm not really sure what you are saying. Regardless, DFS doesn't track moves... and again, the issue doesn't have really have anything to do with DFS.

---

Please don't get too hung up on my individual case... I just provided a lot of background illustration to show that there is an actual (not theoretical) need to be able to control where the sync.ffs_db files are created and stored. And so far I don't have a good work around, Zenju's suggestion won't work in our particular case.

[wm-sf]

> As a casual reader I think you want mirror as the main FFS mode but also need moved files detection. In your case it is looking like these two options aren't going to work together.

Mirror and moved files detection work just fine together. The issue isn't the functionality of FFS, its where the sync.ffs_db files are being created.

> Q: if you had a better way (independent of FFS) of dealing with moved stuff wouldn't you be able to turn that bit of FFS off and your DFS problem would be solved?

My problem isn't really anything to do with DFS, I'm just trying to explain why it is problematic that I can't control where the sync.ffs_db files are being stored. You are correct that if sync.ffs_db files don't show up where end users can see them that I don't have a problem.

> i.e. isn't the "home" issue why DFS isn't tracking the moves rather than why FFS can't track them the way you'd ideally like?

Something may be lost in translation here... I'm not really sure what you are saying. Regardless, DFS doesn't track moves... and again, the issue doesn't have really have anything to do with DFS.

---

Please don't get too hung up on my individual case... I just provided a lot of background illustration to show that there is an actual (not theoretical) need to be able to control where the sync.ffs_db files are created and stored. And so far I don't have a good work around, Zenju's suggestion won't work in our particular case.

I am saying that your case is so specific Zenju can reasonably ignore it (I don't know if he accepts funding for specifics).