Option TO STOP DELETING FILES FROM ABSENT DRIVE

[AlbertX]

Hi,

I have been using freefilesync for a long time and I am even a premium customer or whatever is called for donors.

As much as I like the software there is this big flaw that has caused me so many headaches.

I have 2 PCs connected on the network and the both have 1 12 TB drive that is synchronized between them.

The idea here is that Computer A is the main one and computer B the secondary one.

Computer A has priority so only Computer B runs freefilesync with these rules:

New Files: Anything new on computer A is copied to B and any new file found on B is copied to A
Any file Newer on either A or B is updated on the other one
and Any file deleted on A is deleted on B (but no file Deleted on B is deleted on A).

So essentially this assures me that A will have the most up to date data but it won't get anything deleted that was deleted on B.

I put freefylesync batch that monitors any changes on Computer B, so if I have made any changes on A it will normally be passed across to B.

The problem and this has happened 3 times already and I am so tired because resync for 12 TB on a network takes like 2 days.

Is that if Computer A turns off or freezes then Of course computer B can't find the drive and then instead of doing NOTHING because it clearly can't do a comparison since the drive is missing it decides that NO NO NO, Since I can't find any files on the drive I should DELETE EVERYTHING!!!

And so Drive in computer B is cleaned up.

Surely there must be a way for freefilesync to be more intelligent and realize that the Drive from computer A is NOT available and since no comparison can actually happens instead of just assuming there are no files and delete everything JUST DO NOTHING AT ALL.

Please is there any solution for this?

[xCSxXenon]

That is very very odd because FFS already accounts for this. If a location isn't available for access, such as a network drive that is offline/unreachable, FFS simply won't sync anything and logs it as an error. Can you describe your environment? which computer is sharing its 12TB drive over the network?
And when you say "I put freefylesync batch that monitors any changes on Computer B", do you mean RealTimeSync? FFS doesn't have any monitoring capability

[AlbertX]

Hi, Yes, I meant realtimesync for that.

Computer B has a local 12 TB drive and it sees a 12 TB Network Drive from Computer A.

Last Night everything was ok, I didn't Realize Computer B was On, turned computer A went to sleep and again all my data from Computer B 12 TB is GONE!!!

This has happened 3 times already and I know 100% that is caused by freefilesync or the realtimesync

[xCSxXenon]

Please take a screenshot of the entire FFS window on computer B. Make sure we can see the paths, and attach it here for inspection. I use FFS and RTS with network drives daily. If a location is offline, RTS simply suspends until it comes back online

[AlbertX]

I don;t think I can upload the entire screenshot of it since I never got one, I can upload the log file where is deleting every single folder

[xCSxXenon]

I was just wanting to see what it looked like. A log will also be helpful

[AlbertX]

All the files that have been deleted that you can see in the logs are in my source drive as we speak there is no reason for FFS to have deleted them.

[xCSxXenon]

So based on your description, R: is the drive in computer B, and S: is the drive in computer A. Is that correct?
The logs really don't make sense. It shows S: as not being available, but then there are plenty of files on S: being updated. There are some that are updated twice for some reason also, such as
"S:\Emulation\LaunchBox\Emulators\Dolphin\Sys\Themes\Clean Emerald\assembler_assemble@4x.png"
But then there is also a ton of Access Denied errors, but these shouldn't be possible if S: is unreachable because computer A is off. It can't deny access if access isn't possible.
I'm also confused why "S:\Emulation\Games\Nintendo DS" is listed as not available if "S:" is the location selected for syncing.
This also makes no sense: "Cannot write file "S:\Delete.0.sync.ffs_lock""
I have no clue where the 'Delete.0' part is coming from.

Are both of these computers running Windows? How are you sharing the volume in computer A, the built-in Windows sharing utility?

[AlbertX]

Yes, I shared the Drive on computer A I added it to computer B mapping it and that is all

[Zenju]

A few observations:

The volume "S:\" apparently did "exist" during comparison, otherwise the subfolders would not have been found.

Some folders failed scanning with "Cannot read directory". None of these failures lead to file deletions and they were properly skipped.

On the other hand, the folders that did lead to file deletion, e.g.
"S:\Emulation\Games\3DO Interactive Multiplayer" were in fact found and scanned without errors, otherwise these would have been logged. The fact that the target file was deleted e.g.

Deleting file "R:\Emulation\Games\3DO Interactive Multiplayer\20th Century Video Almanac (USA).ccd"

implies that "S:\Emulation\Games\3DO Interactive Multiplayer" was scanned, found empty, and no errors were reported at all by the S:\ network share.
At that time, e.g. Windows Explorer would also have found and displayed the folder as empty.

Why this (temporary) false information was returned is the big question.

[AlbertX]

That is what I am trying to figure out.
Because this is happening with freefilesync.

Somehow is reporting the file is not there which makes no sense because I can see the file is there

[Zenju]

All these ERROR_ACCESS_DENIED are also wery weird. When the issue really was about network connectivity these should be something like ERROR_NETNAME_DELETED.

Perhaps access control is somehow responsible for these fake empty folders:
https://sharepoint.stackexchange.com/questions/98360/users-getting-empty-folder-instead-of-access-denied

E.g. perhaps access to "S:\Emulation\Games\3DO Interactive Multiplayer" is temporarily denied for some reason, so scanning "S:\Emulation\Games\" simply skips the "3DO Interactive Multiplayer" without returning an error, as it should.

[AlbertX]

Yeah but it ends up deleting files, I have no issues with skiping what I don't understand is why delete the files. This will force me to not allow free file sync to delete anything

[Zenju]

Sure, the problem is how to detect a fake empty folder from a real empty folder when there are no error messages.

[AlbertX]

Of course I understand that, the question is why is freefile sync seeing as empty, what is causing the behaviour

[Zenju]

Access control glitch, most likely:
viewtopic.php?t=11697&p=45400#p45395

[AlbertX]

that link doesn't seem to work

[Zenju]

In case you're able to, a trace file that covers the FFS scanning phase at a time where files are not showing up, although you would expect them to, would help to clarify if it's possible to detect this situation (albeit its a bit clutching at straws, because I don't think it is)
https://freefilesync.org/faq.php#trace

[xCSxXenon]

In essence, this isn't a problem with FFS itself, rather an issue in the underlying system is presenting itself when FFS runs

[therube]

When the issue really was about network connectivity these should be something like ERROR_NETNAME_DELETED.

Those are in there:

12:54:29 Info: Cannot read directory "S:\Emulation\Games\Nintendo Entertainment System".
ERROR_NETNAME_DELETED: The specified network name is no longer available. [DirReaderPlus::readDir: c000020c]
 -> Automatic retry
12:54:29 Info: Cannot read directory "S:\Emulation\Games\Atari ST".
ERROR_NETNAME_DELETED: The specified network name is no longer available. [DirReaderPlus::readDir: c000020c]
 -> Automatic retry
...


[therube]

(
(elsewhere, outside of FFS)

all of a sudden i'm hit with ongoing messages like:

CreateFile(The system cannot find the path specified.3) : S:\III\...


this, while copying files to it, to S:, it went "offline",
bad drive or bad (external) box the drive is in, or... ?

in any case, the drive was there, initially, & then, at some point, it was not
(spooky, on this oddly warm 70.F Halloween night ;-))

so yes, had i been mirroring... (no telling, cause i don't mirror)
)

[Zenju]

CreateFile(The system cannot find the path specified.3) : S:\III\...


this, while copying files to it, to S:, it went "offline",

This can happen due to hardware failure, e.g. a flaky USB cable, or loose contact.
I'm sure this can also happen when the USB port can't handle sustained high load (low-quality chips!?):
I've seen these disconnects on some USB ports of my PC, but not on others.

[AlbertX]

As mentioned before this is a network dirve.

However the funny thing is I have spent the last 3 days transferring all the data (12TB) from the original Disk back to the one erased by freefilesync using teracopy and I had 0 errors, literally 0, no skipped files, no network issues, nothing.

However I have to say that I was trying to do the same using fastcopy and I did get errors like the one on freefilesync about the file "not being there anymore"

So there is something weird somewhere in the way each software handles copying files, because teracopy has given me 0 errors in 3 days while fastcopy will give me error 10 minutes in.

[therube]

This can happen due to hardware failure, e.g. a flaky USB cable, or loose contact.

My M/B (seemingly my M/B) has been acting up of late (... thinking it is on its way out - totally).
Appears to be particular SATA connector(s) on my M/B.
1 seems to have died entirely.
When I plug into another, that causes yet a different drive to vanish from my BIOS (hence system).
Unplug that connector & that vanished drive is back.
Got things to where I believe I'm back (at least for the time being).
Had an add-in eSATA connector & exiSting M/B eSATA connector, & also a separate bank of SATA connectors.
Plugged my external into the M/B eSATA, & the internal drive into one of the separate SATA connects & successfully transferred 4 TB from one to the other.

(In between I tried switching out cables & different SATA connectors, but in all cases when I plugged in to particular SATA connectors, I ran into issues.

Computer has been rock solid from day 1, until more recently. It's close to a decade old.
I kind of have trepidation about 11, which I'm sure I'll loathe ;-).)

[therube]

I periodically, haphazardly, compare files for "exactness".
There have been times that files did not compare - which I find very odd.
And I make notes (haphazardly ;-)) of such occurrences.

And if I come across something like that, my first thought is, "the drive is dying", or there has been "bit rot" or ? whatever.

Anyhow, after rearranging drives/connectors (above), I now go & check said files that did not compare - before - & they now do compare, exactly.

So my supposition that it was "old" drives that were failing, was not correct. The drive is actually good, but it was parts between the keyboard & the drive that were failing.

("Old". On the order of 12 years old ;-).)