Overwrite new files with old ones: bug?

Get help for specific problems
Posts: 13
Joined: 7 Feb 2017

4world

I've just started using this program. Fortunately, I noticed in preview after doing a compare that it was going to copy old files over new ones wiping out days worth of my work!! I would have killed myself for losing so much work.

The screenshot is attached where it says Right Side is Newer yet you can see the arrow pointing right.

I can't believe it's a bug in your program since it has so many positive reviews so I must be doing something wrong. Would someone help please?
FreeFileSync- Copying to wrong side, SML.jpg
FreeFileSync- Copying to wrong side, SML.jpg (140.62 KiB) Viewed 8762 times
User avatar
Site Admin
Posts: 7211
Joined: 9 Dec 2007

Zenju

The "two way" variant propagates any changes: If both sides are equal and you overwrite a newer file with an older one on one side, FFS will also propagate this change.
Posts: 13
Joined: 7 Feb 2017

4world

"Two-way variant" -- I assume it's for syncing both ways (i.e. if files on A change, then they are copied to B, and vice-versa). And this is likely the most common way people would be using it.

Regardless, it is totally unacceptable for a software like this to copy old files over new. I have never heard of such a thing regardless of what options you select. Unless you clarify further, I am tempted to now give this program a 1-star review so others can be warned of this problem.
User avatar
Site Admin
Posts: 7211
Joined: 9 Dec 2007

Zenju

Copying old file over newer happens every time when you revert to a former state. Again, FFS just propagates changes, so in the example above *you* deliberately overwrote newer files with older ones. FFS sees this change on the left side and syncs accordingly.
Posts: 13
Joined: 7 Feb 2017

4world

My respectful reply Please:

I did NOT overwrite newer files with older ones. I just did a simple comparison between my laptop and desktop computers -- and this was the first time. **YOU** need to see the screenshot I have attached above which shows the dates of a file on both sides (the yellow pop-up which shows when you hover the cursor as shown above). Your program is showing that the file on the right side is newer but the direction of copying is left to right.

Fortunately, I did not press "Synchronize" blindly, otherwise it would have overwritten the newer files with the old ones from the left.
User avatar
Site Admin
Posts: 7211
Joined: 9 Dec 2007

Zenju

Maybe you manually changed the sync direction? If two folders are compared for the first time with "two way" variant (= no database files existing yet), FFS will default to "overwrite old with newer files". BTW what version are you using?
Posts: 6
Joined: 5 Feb 2017

evm

WOW...
4world,
Is it stable situation? I mean can it be repeated again? If yes, maybe you can make the video?
User avatar
Site Admin
Posts: 7211
Joined: 9 Dec 2007

Zenju

If yes, maybe you can make the video? evm, 07 Feb 2017, 14:07
Agree. Can't believe that FFS will set the wrong sync direction when comparing two folders *for the first time*. Also technically, such a behavior cannot happen by accident.
Posts: 13
Joined: 7 Feb 2017

4world

I am usually far more careful than an average non-tech user since I am myself a hardware and embedded control software engineer for decades and we, by training, are extremely watchful of details like good yourself.

Since I could not believe my own eyes, I repeated the test probably at least 3-4 times (on two different occasions), without of course pressing the Synchronize button, and stared (literally gazed) at the screen since it didn't make logical sense that an application so highly rated could do such a murderous thing.

I eventually gave up and started looking for other apps that could do the job (and finally found one). I wish I had taken a video.

But guys, more important than the video, the BASIC FUNDAMENTAL question remains -- why under any circumstances WHATSOEVER and EVEN ONCE in a lifetime, when the direction was set to "Two-way" sync, would FFS do such catastrophic a thing. This is visible from the screenshot posted above (I hope you believe me it's not doctored) -- please see the direction of the sync in that pic. I tried to post the original screenshot but your website has a very low limit for attachments so I had to crop it and reduce the resolution. If you want, I can email you the original high-res screenshot (give me your email ID).

Friends, I cannot afford to take chances with my precious data even once in a lifetime -- this scared the daylights out of my eyes when it was about to obliterate probably 1-2 weeks worth of hard work! I have just lost faith in this application and I hope other users take note and be careful.
Posts: 13
Joined: 7 Feb 2017

4world

Also, if you have written this application or you are a software engineer, you probably know that bugs can occur under very special conditions that are somewhat uncommon in usual practice (say the user set some conditions for the sync using date-time, size and archive bit and set/reset something else somewhere else, which created that condition for the bug). Bugs can thus be very elusive. BOTTOM LINE is this -- the software should not under any circumstances, without warning the user at least twice, overwrite new files with old ones.

I have submitted an irrefutable piece of evidence above (the screenshot) and I'm ready to send you the high-res pic. of the same. If I were you, I would not need further evidence.
User avatar
Site Admin
Posts: 7211
Joined: 9 Dec 2007

Zenju

The screenshot merely shows that sync directions are set so that new files are overwritten by old files. This is *not* a bug in general. The only case this situation would be a bug is if this were the screen after a fresh first compare with "two-way".

If there are steps how to reproduce this behavior, I'm more than willing to have a closer look. But I'll be clear that I highly doubt that such a fundamental bug even exists. It's likely that it was a user error of some kind, maybe accidentally setting the sync direction via keyboard (ALT+right). Or the left side was updated after an initial sync had already run. But without further evidence this is all guesswork.
Posts: 13
Joined: 7 Feb 2017

4world

Again, respectfully submitting to you Zenju --

1. SCREENSHOT. I am referring to the "Synchronize" button on the top right side. You are probably referring to the arrows along the middle that show the direction of copying. Please see the Synchronize button which has small text below it that says "Two way".

2. CLEARER EXPLANATION.
Allow me to explain even more clearly please:

IF--
a) Your Sync button is showing "Two way" sync,
b) The Compare button is showing "File time and size", and
c) The file dates are as shown in the screenshot the left files are older, the right newer),

THEN --
Why is copying direction reversed???

It shouldn't matter what I did before or how I got to the above point since the proof is visible above in the failed logic (as long as you believe me that I didn't tamper with the screenshot).

Am I missing something or are you missing my point?

I won't spend further time explaining this to you as I have already spent enough and you seem to not understand my basic point-- now even explained in a programmer's language above. Good luck and thanks for interacting with me so far. I hope this app doesn't cause damage to someone's important work as it was about to do so with me.
User avatar
Site Admin
Posts: 7211
Joined: 9 Dec 2007

Zenju

Again, the screenshot does not proof a bug. The situation shown in the screenshot can happen legitmately in certain cases. If you think it is a bug, you need to provide proof, e.g. steps to reproduce or be able to reproduce your test case. In this case I'd have a closer look.
Posts: 13
Joined: 7 Feb 2017

4world

I feel you are wrong in your assessment of the clear IF-THEN statement I provided above.

I bet if people were to vote on such a thing, they would side with my assessment.

Thanks
User avatar
Posts: 2451
Joined: 22 Aug 2012

Plerry

I think the main issue is about the notion of "File time", versus running a two-way sync, and particularly the purpose/use of the ffs_db-file(s) in case of two way syncs.

At the risk of getting burned, or adding to the confusion rather than reducing/resolving the matter:
As described for the Two-way sync setting :
"Identify and propagate changes on both sides. Deletions, moves and conflicts are detected automatically using a database."
I'll try and explain as I understand FFS to work for a simple 1:1 two way sync (Zenju, please correct me if I am wrong)
The ffs_db file in each left or right sync location contains all file sizes and modification dates as per the end of the last FFS sync run on that left- respectively right-location. When now running a comparison, the content of each location is first compared to the content of its (local) ffs_db file. Any changes to the locations content compared to what is stored in the local ffs_db-file is considered to be more recent than the last FFS sync was run, even if the modification date of a file that has changed is earlier than the last time FFS sync was run on that location.
If the above is the case in 4world's left location, and the content of 4world's right location has not changed (compared to what is stored in its ffs_db file), the file change date/time in the left location is more recent than that in the right location and thus needs to propagate left to right, even if the right-side file has a more recent modification time than the left-side file. And this seems to be the only correct two-way sync action !!!
If the content of both the left and right location differs from what is stored in their respective ffs_db-files, this constitutes a conflict, which needs to be resolved manually.

So, for a two-way sync the definition "File time" needs to be interpreted as "changed since the last sync", rather than "having the most recent modification date/time".

I can understand that this might be ground for confusion.
Having a vote on the topic is not very useful; winning a vote does not make anyone "right" or "wrong".
User avatar
Site Admin
Posts: 7211
Joined: 9 Dec 2007

Zenju

@Plerry: You are absolutely correct.
Posts: 13
Joined: 7 Feb 2017

4world

Plerry,

Thanks for participating in the discussion as your comments have added great value to the debate.

1. I've understood the copying algorithm you describe but as engineers, we must design systems that are idiot-proof and fail-safe. This algorithm is obviously neither; because it was about to destroy newer files (assume I was an idiot and pressed the sync button, so it would have failed in a "non-fail-safe" way destroying new data).

When the app compared the left side file times with its own ffs_db, and determined that those files had been changed later, a more intelligent algorithm (fail-safe) would have additionally seen that there was a conflict in dates on the two sides, and thus would have raised a red flag (say another type of conflict flag) to save the user's data. The program must assume (IMO), that the user's 1:1-sync computers are regularly synced and that the local times on each machine are within seconds of each other (that is the assumption that will make the software fail-safe), and thus at least raise a red flag; it should not under any circumstances overwrite new files without a double confirmation from the user. At least, I would have implemented this code this way (that's how I was trained -- we used to design sophisticated medical instruments and could not afford a single failure no matter what).

2. "...winning a vote does not make anyone "right" or "wrong""

I am also a firm believer in the above line as I'm sure a lot more people today are beginning to believe it as well (having seen how supposedly democratic nations can elect strange persons to head their nation.) :-)

I had exhausted all ways to convince Zenju and also thought that most people who might vote on a subject like this (if it were ever to happen) would be technically competent. I knew it was not going to happen.
Posts: 41
Joined: 10 Jul 2016

JustAnotherUser

I absolutely see it the same way was as Plerry does.

@4world:
In the matter of a 2 way sync tool, FreeFileSync absolutely behaves correct.
If the user replaces a newer file by an older one, it is his own decision to do that.

Your point seems to be, that FFS should be able to protect the user from himself.
But that is more or less a feature of a backup program.

Edit:
...and also thought that most people who might vote on a subject like this (if it were ever to happen) would be technically competent. I knew it was not going to happen.4world, 08 Feb 2017, 16:44
Your words are a bit offending. You seem just to accept your own point of view.
I'm also a software developer, since decades, so "thanks for the flowers"...
Posts: 13
Joined: 7 Feb 2017

4world

...and also thought that most people who might vote on a subject like this (if it were ever to happen) would be technically competent. I knew it was not going to happen.4world, 08 Feb 2017, 16:44
I'm also a software developer, since decades, so "thanks for the flowers"... JustAnotherUser, 08 Feb 2017, 16:51
@JustAnotherUser
CLARIFICATION regarding your sarcastic statement --
I'm sorry about my English. "I knew it was not going to happen." meant that I knew nobody would put such a thing for voting, and NOT that software engineers will vote incorrectly.
Posts: 41
Joined: 10 Jul 2016

JustAnotherUser

No problem, back to topic. ;-)
Posts: 13
Joined: 7 Feb 2017

4world

Your point seems to be, that FFS should be able to protect the user from himself.
But that is more or less a feature of a backup program. JustAnotherUser, 08 Feb 2017, 16:51
That's a difference in design philosophy like I stated. My design philosophy is obviously different than yours.
Posts: 41
Joined: 10 Jul 2016

JustAnotherUser

So the facts of this discussion seem to be:
- FFS detects a conflict for example, if a file on both sides has been changed.
- 4worlds suggestion is, as far as I understood, that the situation, when a newer file would be overwritten by an older one, should also be rated as a conflict.

Am I right, 4world?

So maybe this could be implemented as a compare option for a further release of FFS, Zenju?
Posts: 6
Joined: 5 Feb 2017

evm

The easy way to reproduce the situation:
1. Create file in the left folder (version 1)
2. Sync it to the right folder (FFS just copies it to right side, OK)
3. ZIP left folder and save this archive in some place
4. Edit file in the left folder (version 2)
5. Sync it (FFS copies file version 2 to the right, OK)
6. UNZIP archive to the left folder. You get the file version 1, with older creation time
7. Try to sync. Now you will see from-left-to-right direction despite right file (is version 2) is newer than left one (version 1).

I agree that is not very obviuos behavior. But in other side it works according the algorithm - the left side file was [re]created in earlier version by the user and need to be copied to the right side.
User avatar
Site Admin
Posts: 7211
Joined: 9 Dec 2007

Zenju

So maybe this could be implemented as a compare option for a further release of FFS, Zenju? JustAnotherUser, 08 Feb 2017, 17:29
The whole discussion is just about a specific manifestation of the more general problem of the user accidentally corrupting his data. The proper solution is to keep versions of deleted/overwritten files.
Posts: 41
Joined: 10 Jul 2016

JustAnotherUser

Yep, you're right, that's also a possible solution.
Posts: 13
Joined: 7 Feb 2017

4world

The easy way to reproduce the situation:
... evm, 08 Feb 2017, 18:09
Brilliant example evm. Thanks.
But in other side it works according the algorithm - the left side file was [re]created in earlier version by the user and need to be copied to the right side. evm, 08 Feb 2017, 18:09
If the app is designed only for engineers/techies, then maybe it's okay. But hey, even tech people use complex types of file operations (you described one type) and then may forget to watch out, and then kaboom! I don't see why the app cannot warn the user in a potentially dangerous situation -- it's as simple as that.

By now, I have used at least 6-7 different syncing apps but never came this close to data destruction by the app. I see conflict flags by syncing apps all the time and resolve each conflict manually but that's far better than losing precious data.
Posts: 13
Joined: 7 Feb 2017

4world

The whole discussion is just about a specific manifestation of the more general problem of the user accidentally corrupting his data. Zenju, 08 Feb 2017, 18:46
Dear Zenju,

First off, I want to thank you and all other developers/users who have contributed to make this product available to the public free of cost -- it is the most noble work humans can engage in when they serve others selflessly. My critique of this product is in no way intended to hurt anyone or cause disrespect or diminish my preceding appreciation. Also grateful to you for creating and supporting this forum, due to which we users are able to at least voice our concerns and get meaningful feedback.

Second, I beg to disagree again with your assessment "... user accidentally corrupting his data." All I can say after all my explanations thus far is that this is likely the NIH syndrome that I've come across so many times over the decades. My comment may be seen coming from a different design philosophy where engineers try to make products as intelligent as possible even sometimes exceeding normal human intelligence -- which makes them better, safer and more friendly. Your statement that is putting the blame on the users feels wrong to me because when I have designed products, I have sought the most damning critique and have seldom cared for appreciation (that's what makes products better and better).

Won't argue further that a simple warning to user would have made this product usable for me... I have already moved on to another one. Thanks again
User avatar
Posts: 2451
Joined: 22 Aug 2012

Plerry

@evm
...7. Try to sync. Now you will see from-left-to-right direction despite right file (is version 2) is older than left one (version 1).evm, 08 Feb 2017, 18:09
I believe this should read
....7. Try to sync. Now you will see from-left-to-right direction despite right file (is version 2) is newer than left one (version 1).
Posts: 6
Joined: 5 Feb 2017

evm

Plerry,
Sure, you are right, right file v.2 is definitly newer than left one v.1 :)
Sorry for mistake.
I have corrected my post, thanks!
Posts: 6
Joined: 15 Dec 2016

fishcad

This is happening with both v10.13 and v10.14.