Network error causes mass file deletion

[beddhist]

I have long used FFS as an automatic backup of most of my HD data to a network drive and this has served me well for years. However, some months ago I was stunned to discover that several folders had disappeared from My Documents. I was able to copy them back from the normal backup destination. They were always the same folders. I have now some clues from the log file:

|13/03/2015 - BigBatchRun: Synchronization stopped
|
| Items processed: 1,554 (1.34 GB)
| Items remaining: 8,208 (0 bytes)
| Total time: 03:33:18
|_________________________________________________

[16:57:45] Info: Starting comparison
[16:58:54] Error: Cannot enumerate directory "\\BEDDHISTSAP\Expansion Drive\Backups\Peter-NEC\Data\Documents\Drivers\AUDIO\Config".
Error Code 64: The specified network name is no longer available. (readDir)
[16:59:02] Warning: The following folders are significantly different. Make sure you are matching the correct folders for synchronization.
C:\Users\Peter\Documents\ <->
\\BEDDHISTSAP\Expansion Drive\Backups\Peter-NEC\Data\Documents\
[16:59:02] Info: Synchronizing folder pair: [<- Two way ->]
C:\Users\Peter\Downloads\
\\BEDDHISTSAP\Expansion Drive\Backups\Peter-NEC\Data\Downloads\
[16:59:02] Info: Synchronizing folder pair: [<- Two way ->]
C:\Users\Peter\Documents\
\\BEDDHISTSAP\Expansion Drive\Backups\Peter-NEC\Data\Documents\
[16:59:02] Info: Moving file "C:\Users\Peter\Documents\Untitled.skp" to "\\BEDDHISTSAP\Expansion Drive\Backups\Peter-NEC\OldVersions\"
[16:59:06] Info: Moving folder "C:\Users\Peter\Documents\Tax" to "\\BEDDHISTSAP\Expansion Drive\Backups\Peter-NEC\OldVersions\"
[16:59:07] Info: Moving folder "C:\Users\Peter\Documents\Teaching" to "\\BEDDHISTSAP\Expansion Drive\Backups\Peter-NEC\OldVersions\"
[16:59:07] Info: Moving folder "C:\Users\Peter\Documents\ThunderbirdProfile" to "\\BEDDHISTSAP\Expansion Drive\Backups\Peter-NEC\OldVersions\"
[16:59:45] Info: Moving folder "C:\Users\Peter\Documents\Tmp" to "\\BEDDHISTSAP\Expansion Drive\Backups\Peter-NEC\OldVersions\"
[16:59:45] Info: Moving folder "C:\Users\Peter\Documents\Trains" to "\\BEDDHISTSAP\Expansion Drive\Backups\Peter-NEC\OldVersions\"
[17:04:23] Info: Moving folder "C:\Users\Peter\Documents\Wind power" to "\\BEDDHISTSAP\Expansion Drive\Backups\Peter-NEC\OldVersions\"
[17:04:24] Info: Moving folder "C:\Users\Peter\Documents\XP" to "\\BEDDHISTSAP\Expansion Drive\Backups\Peter-NEC\OldVersions\"
[20:31:04] Error: Synchronization stopped

For some reason FFS could not see these folders, assumed they had been deleted on the destination and proceeded to remove them. I killed it once I could see what it was doing.

Restarting the batch job made it resume removing the files from XP. While it was moving a subfolder in XP with over 3000 files it ran very slowly and bogged down an explorer window I had open on the network drive. Killed it again.

Now that I have copied the folders back it is working again.

I'm wondering whether there is a timeout issue here. The network drive powers down after some idle time and it takes 20-30 secs for it to come back online when it is accessed.

Regards,
Peter.

[Zenju]

The network error at the top "Cannot enumerate directory" is unrelated to the file deletions further below. In general a network drop or other error while scanning a directory does not lead to file deletions, because FreeFileSync skips the corresponding folders.

Looking at one of the deleted files as an example:
[16:59:02] Info: Moving file "C:\Users\Peter\Documents\Untitled.skp"

it seems that \\BEDDHISTSAP\Expansion Drive\Backups\Peter-NEC\Data\Documents\Untitled.skp was missing on the network drive.

[beddhist]

Thanks for your reply. This kind of error (files missing, folders inaccessible) happen sometimes with this router attached drive. As I said, it appears to be related to the drive going to sleep and then taking some time to spin up (plus the Linksys router seems very slow overall). That, I guess, leads to timeouts in Windows. I can hear the drive start every time I modify something in the monitored folders, so most of the time all is well.

However, once the above error occurs, FFS seems to tie itself into a knot and when the drive becomes available it goes ahead and removes folders to old items. Somehow, it seems to interpret the network drive missing as the folders having been removed. When the drive comes online it removes the folders from my hd. At the next scan it finds the folders on the network drive, but the folders on the HD have been removed, so it now removes the folders on the drive as well.

There is some guesswork here on my part, but this has happened several times in the past and it always seems to involve the same folders, all in the same area of the alphabetically sorted folders.

Am I making sense?

Thanks,
Peter.

[Zenju]

The only situation where FreeFileSync would delete files is when they are don't existing when scanning the source directory *and* there also was no error either during scanning or reading the specific file item. FreeFileSync takes great care to avoid any race condition, e.g. does not repeat folder existence checks.
So the only chance for FreeFileSync to unwarrantedly delete files from the target side is when the source drive failed to deliver correct scan results but still incorrectly reported success. This would be a driver or hardware issue then.
BTW which version of FreeFileSync are you using?

[beddhist]

Oh great, more bugs, this time on this forum. I just posted a long reply, was asked to log on after posting, then returned to this same page with my post gone to the bit bucket. :-(

I'm using v 6.14, have just downloaded 6.15 now. Two days ago exactly the same thing happened again, the same files an folders got removed after network error:

[16:24:18] Info: Starting comparison
[16:25:29] Error: Cannot enumerate directory "\\BEDDHISTSAP\Expansion Drive\Backups\Peter-NEC\Data\Documents\Saved web sites\Our house on Trademe\images.trademe.co.nz\photoserver\thumb\34".
Error Code 64: The specified network name is no longer available. (readDir)
[16:25:34] Warning: The following folders are significantly different. Make sure you are matching the correct folders for synchronization.
C:\Users\Peter\Documents\ <->
\\BEDDHISTSAP\Expansion Drive\Backups\Peter-NEC\Data\Documents\
[16:25:34] Info: Synchronizing folder pair: [<- Two way ->]
C:\Users\Peter\Downloads\
\\BEDDHISTSAP\Expansion Drive\Backups\Peter-NEC\Data\Downloads\
[16:25:34] Info: Synchronizing folder pair: [<- Two way ->]
C:\Users\Peter\Documents\
\\BEDDHISTSAP\Expansion Drive\Backups\Peter-NEC\Data\Documents\
[16:25:34] Info: Moving file "C:\Users\Peter\Documents\Untitled.skp" to "\\BEDDHISTSAP\Expansion Drive\Backups\Peter-NEC\OldVersions\"
[16:25:38] Info: Moving folder "C:\Users\Peter\Documents\Tax" to "\\BEDDHISTSAP\Expansion Drive\Backups\Peter-NEC\OldVersions\"
[16:25:39] Info: Moving folder "C:\Users\Peter\Documents\Teaching" to "\\BEDDHISTSAP\Expansion Drive\Backups\Peter-NEC\OldVersions\"
[16:25:39] Info: Moving folder "C:\Users\Peter\Documents\Tmp" to "\\BEDDHISTSAP\Expansion Drive\Backups\Peter-NEC\OldVersions\"
[16:25:39] Info: Moving folder "C:\Users\Peter\Documents\Trains" to "\\BEDDHISTSAP\Expansion Drive\Backups\Peter-NEC\OldVersions\"
[16:32:06] Info: Moving folder "C:\Users\Peter\Documents\Wind power" to "\\BEDDHISTSAP\Expansion Drive\Backups\Peter-NEC\OldVersions\"
[16:32:07] Info: Moving folder "C:\Users\Peter\Documents\XP" to "\\BEDDHISTSAP\Expansion Drive\Backups\Peter-NEC\OldVersions\"
[16:43:58] Error: Cannot create directory "\\BEDDHISTSAP\Expansion Drive\Backups\Peter-NEC\OldVersions\XP\I386".
Error Code 53: The network path was not found. (CreateDirectory)
[16:43:58] Info: Moving folder "C:\Users\Peter\Documents\XP\I386" to "\\BEDDHISTSAP\Expansion Drive\Backups\Peter-NEC\OldVersions\"
[16:43:58] Error: Cannot create directory "\\BEDDHISTSAP\Expansion Drive\Backups\Peter-NEC\OldVersions\XP\I386".
Error Code 53: The network path was not found. (CreateDirectory)
[16:43:58] Info: Moving file "C:\Users\Peter\Documents\XP\I386\mmc30r.dl_" to "\\BEDDHISTSAP\Expansion Drive\Backups\Peter-NEC\OldVersions\"
[16:43:58] Info: Moving file "C:\Users\Peter\Documents\XP\I386\MMCBASE.DL_" to "\\BEDDHISTSAP\Expansion Drive\Backups\Peter-NEC\OldVersions\"

[... lots more like this ...]

[16:43:58] Info: Moving file "C:\Users\Peter\Documents\XP\I386\MOVE_L.CU_" to "\\BEDDHISTSAP\Expansion Drive\Backups\Peter-NEC\OldVersions\"
[16:43:58] Error: Cannot create directory "\\BEDDHISTSAP\Expansion Drive\Backups\Peter-NEC\OldVersions\XP\I386".
Error Code 53: The network path was not found. (CreateDirectory)
[16:43:58] Info: Moving file "C:\Users\Peter\Documents\XP\I386\MOVE_M.CU_" to "\\BEDDHISTSAP\Expansion Drive\Backups\Peter-NEC\OldVersions\"
[16:43:58] Error: Cannot create directory "\\BEDDHISTSAP\Expansion Drive\Backups\Peter-NEC\OldVersions\XP\I386".
Error Code 53: The network path was not found. (CreateDirectory)
[16:43:58] Info: Moving file "C:\Users\Peter\Documents\XP\I386\MOVE_R.CU_" to "\\BEDDHISTSAP\Expansion Drive\Backups\Peter-NEC\OldVersions\"
[16:43:58] Error: Cannot create directory "\\BEDDHISTSAP\Expansion Drive\Backups\Peter-NEC\OldVersions\XP\I386".
Error Code 53: The network path was not found. (CreateDirectory)

[... eventually, the log file is truncated with this line: ]

[...]

[beddhist]

I just discovered that my problem has become several order of magnitudes bigger: while until now I could copy the folders back from OldVersions seemingly with all files intact, now I have 8 versions of most files with the names having time stamps appended. At this point I can't see how I can batch rename the files, so it looks like I have to manually rename and move thousands of files.

If anybody can see how I can semi-automate this process - please help.

Regards,
Peter.

[Zenju]

> Oh great, more bugs, this time on this forum. I just posted a long reply, was asked to log on after posting, then returned to this same page with my post gone to the bit bucket. :-(

Feel free to complain at the sourceforge support forum to hopefully get this bug fixed at some time (see "support request" links below). You're not the only one...

> I'm using v 6.14, have just downloaded 6.15 now. Two days ago exactly the same thing happened again, the same files an folders got removed after network error:

Looks the same like last time: The directory traversal error at the top is unrelated to the subsequent move operations, so some folders were not found on the network, but either erroneously no failure was reported, or they really were not there.

> now I have 8 versions of most files with the names having time stamps appended.

It seems you've set version naming convention to "time stamp"; the default was "replace" (see sync settings).