FFS creating local directories called FFS YYYY-MM-DD HHMMSS

[mfreedberg]

Hello everybody!
We are seeing an occasional failure in our global FFS deployment where people are seeing the sync process creating a LOCAL folder called "FFS YYYY-MM-DD HHMMSS", so something like "FFS 2013-01-17 120506". These folders look like they are duplicating what would have been copied to the target, but all that copying is happening to that local folder. It seems like FreeFileSync is losing the valid target and is creating a fallback directory, but this is causing some real issues because the next real sync is then going to add that directory to the backup share.

Looking at the code, is there an obvious place or condition that would cause FFS to create a folder with that specific naming convention. We are not using any date or time macros in our directory names, so this is originating from FFS itself.

If you can help us understand under what conditions FFS will create a folder with this naming convention, that would be wonderful - and we can try to figure out how to prevent it (as well as detect and remediate).

[Zenju]

These are intermediate diretories that are created when deletion handling is set to "recycle bin" and should be removed automatically after sync (their content went into recycler, the empty directory is then deleted).
viewtopic.php?t=710

[mfreedberg]

somehow I am losing my entries here (grin)...one more time!
What happens if FFS were to close or quit, or for some other reason, fail to delete those files and remove that directory during a sync event? Would it know to remove those files the next pass or would that now look like a normal real directory that FFS would just keep including in the backup?

[Zenju]

It would look like a normal directory that FFS would sync. This shouldn't be a problem though, since in order to leave these files behind, FFS would have to crash or the network connection be cut.

[mfreedberg]

Ok, so I need to go back now to the users reporting this "error" to see if these folders actually contain files that were pending a move to the Recycle Bin; if they do, I would imagine they are now considered normal directories and we sync them.
We are definitely seeing a minor uptick in this, but that could just be because people are hibernating the system while the sync is running, for example -- they do not really ever see the taskbar icon, it's hidden for most of the run.

Other than showing the details dialog, what other option do we have to show the taskbar icon? It gets hidden after 30 seconds or so, and I can see how clients would have no idea it is runnning.

PS - we are version 5.7, and I can see that we might benefit from an upgrade to 5.10 or later, including performance benefits for scanning directories.

[Zenju]

> people are hibernating the system

Maybe this is somehow related. But on the other hand if hibernation works correctly, all applications, including FFS, should continue as if nothing happened - unless there is some impact on network connectivity. In any case, if FFS is unable to delete these temp directories you should see a corresponding error entry in the log file.

For .ffs_tmp files FFS implements a special rule to mark them for deletion after comparison. Maybe the same should be done for the "FFS timestamp" folder, to cover the allegedly rare cases where FFS fails to clean up.

> It gets hidden after 30 seconds or so

There is no timeout in FFS. The only way to hide the progress dialog is to minimize the window. Maybe the users were accidentally minimizing it? With windows 7 systray icons are hidden by default so for those systems it is a good idea to configure Windows to always show the FFS/RTS tray icons.

[mfreedberg]

If the user hibernates at work, the connection to our backup share is lost, and then when the user comes out of hibernation at home, and FFS picks up again, the network share is of course now no longer available - and this might be leading to some of these issues. The problem is that now FFS has no way to know to clean up this directory after the fact, and I like the idea of a special rule to mark these for deletion or clean up.

Not sure we can configure the system tray to show only a specific icon, but if there is a way to do that, we will certainly look into that.

[Zenju]

> special rule

Currently FFS has a fixed naming convention for these temp directories like "FFS 2012-05-15 131513 [2]", so for a special rule to work we would need two changes: 1. mark these directories, e.g. simply by a .ffs_tmp postfix 2. make sure they are unique on left and right side, else they may be categorized as "equal" on both sides and FFS does not support a sync action "delete on both sides".

> system tray to show only a specific icon

Win 7 allows the user to configure in detail which systray application should be visible and which not. By default all icons are hidden and the user needs to enable those he likes to see. This is a huge source of confusion for FFS users which minimize the progress dialog and think the program just vanished.

[Zenju]

I've implemented the automatic cleanup outlined above:
The temporary directories will be named like "3vkf74fq.ffs_tmp" and be set for deletion after comparison. Here's the new version for testing:

[404, Invalid URL: http://freefilesync.sourceforge.net/FreeFileSync_5.12_beta_setup.exe]

[mfreedberg]

Thank you for the beta code, we will test it starting today and also try this on users who seem prone to seeing this issue occur.
I wonder how best to simulate this for testing? Does this happen when the user has deleted files and then FFS is doing the folder cleanup on the local drive? Would we see the folder being created on the fly and then go away?

Is the test case something like:
1. Delete a large number of files on the client
2. Start the sync and let FFS process the deletions by deleting the files on the target share.
3. Shut down the machine before FFS hits the end of the sync and cleanup process which should leave the temporary folder behind?

[Zenju]

Yes, something like that. The order of steps FFS takes is: 1. the temp directory is created at the time the first to be deleted file is processed, then it is moved to this directory. 2. At the end of processing the folder pair, the content of the temp directory is moved to recycler, then the remaining stub directories are deleted permanently.

So a test case could look like: 1. run FFS until at least one file was shown on GUI as being moved to recycler (it's not yet there, just in the temp dir). 2. cut the network connection or kill the FFS process. This will leave the temp file behind, which a second FFS process should schedule for deletion.

[mfreedberg]

Can you please clarify this so that I can make sure my testing is on target?
"So a test case could look like: 1. run FFS until at least one file was shown on GUI as being moved to recycler (it's not yet there, just in the temp dir).
When you say "on GUI", do you mean the FFS GUI or actually moved to the Recyle Bin in Windows and we see it happen while FFS is running?

I also guess that these deleted files are ones that were deleted by some means OTHER than the user deleting the file in Explorer. Does FFS create these temporary folders for items that I deleted via Explorer and are already in the Recycle Bin or would this apply to items deleted via other means, as in file deletions via CMD window or Office cleaning up temp files? One of the files I can see in this FFS directory is something like "~$My fake filename(2).pptx", which would be something deleted on the client but programmatically.

[Zenju]

> When you say "on GUI",

I meant the time when the progress dialog shows the status message "moving ... to recycle bin".

> Does FFS create these temporary folders for items that I deleted via Explorer

FFS creates a temp dir if there is at least one file deleted or overwritten by FFS itself.

[mfreedberg]

Ah, ok!! That makes sense, I think I can simulate this in our environment and will watch for the "moving...to Recycle Bin" message in the GUI and will try to kill the process or drop the network connection before then and see if I can get these new FFS-owned directories to stay behind and then see if they go after the next sync.

[wpwoodjr]

I'm confused. If I understand the scenario, it's something like:

1. Folder pair is created to mirror local PC client files to network
* FFS runs and syncs a bunch of files from local PC to network
* User deletes a bunch of files on local PC client
+ User hibernates while FFS is running
+ FFS temp file which holds deleted files to be moved to recycler is left behind on local PC

Why would FFS be deleting files on the local PC when they have already been deleted by the user?

[wpwoodjr]

I'm confused. If I understand the scenario, it's something like:

1. Folder pair is created to mirror local PC client files to network
* FFS runs and syncs a bunch of files from local PC to network
* User deletes a bunch of files on local PC client
+ User hibernates while FFS is running
+ FFS temp file which holds deleted files to be moved to recycler is left behind on local PC

Why would FFS be deleting files on the local PC when they have already been deleted by the user?

If it's a 2-way sync and the user has 2 PC's I suppose the user could delete files on one PC and then the recycler issue could pop up on the second PC.