Batch corruption of sync.ffs_db in Version 11.5

[TerenceW]

After upgrading to 11.5, the sync.ffs_db files in a batch task (in a profile) always get corrupted on the overwritten side (right side). I saw a similar issue was reported in the forum, and the answer was maybe it's due to the version difference. This is not the case here. All my computers are Windows 10 running 11.5 release, and I have tried deleting all sync.ffs_db files, but soon errors popped up and said the files are corrupted again (invalid header). No new files are corrupted, so this is not a hard disk issue. If you run Windows disk tools by right-clicking on the disk, check for errors, it will tell you that sync.ffs_db files are damaged. If you ignore the error and continue the sync, chances are new added files are synced to the removable disk, but later only to find it be deleted by the freefilesync software on a new sync task on the outdated computer. I checked the missing file with a hard disk tool. It is recoverable, deleted, not corrupted. I think this is a complication that brought by the sync.ffs_db error. In 11.4, it is normal.

[xCSxXenon]

Does it work in 11.6?

[TerenceW]

Does it work in 11.6?

I updated to 11.6, so far so good. But I don't know whether it is totally fixed, because I didn't see it in the changelog of 11.6. If it occurs again, I'll open a new thread and link this thread.

[Zenju]

I updated to 11.6, so far so good. But I don't know whether it is totally fixed, because I didn't see it in the changelog of 11.6. If it occurs again, I'll open a new thread and link this thread.

Corrupted ffs_db files are an indication of a malfunctioning hard drive, not FreeFileSync. The file itself is written transactionally, so there is essentially zero(*) chance this corruption is from FFS.

*: With one caveat: File formats change and if you are using different FFS versions, one of them being very old, this might explain why FFS sees files created from this old version as corrupted.

[TerenceW]

I updated to 11.6, so far so good. But I don't know whether it is totally fixed, because I didn't see it in the changelog of 11.6. If it occurs again, I'll open a new thread and link this thread.

Corrupted ffs_db files are an indication of a malfunctioning hard drive, not FreeFileSync. The file itself is written transactionally, so there is essentially zero(*) chance this corruption is from FFS.

*: With one caveat: File formats change and if you are using different FFS versions, one of them being very old, this might explain why FFS sees files created from this old version as corrupted.

Thank you for your response and I have already considered this possibility last time, and have tested it.

It is NOT because of a malfunctioning hard drive, because it only has corrupted ffs_db files and not a single "normal" new files would be corrupted when they are copied into the drive. I have also set up monitors and found that, if any "normal" file is found missing under this circumstance, it is marked deleted (by FreeFileSync via an unknown ffs_db misoperation, maybe because of the "corruption") and can be fully restored, while the corrupted ffs_db file CANNOT be restored. Deleting the ffs_db manually and let the software rebuild it would also lead to ffs_db corruption (everytime).

I have also swapped other drives for the test, among my three computers. I never mix-use old versions of FFS because I have the habit of updating everything to the latest when it is available.

When I downgraded FFS to the older 11.4, it never happened again on this drive or any other drives on my three computers. On 11.6 it is also good, so far, but I'm not 100% sure because I have just upgraded to it.

I suspect that there might be some writing error of ffs_db files in 11.5, that tricks the software into thinking that the file is corrupted, and also would misoperate files. This is indeed quite strange, and I cannot tell what causes it. Apparently there were not many cases like this.

[Zenju]

Deleting files as a consequence of a corrupted ffs_db file cannot happen. FFS shows an error and defaults sync directions to copy to the other side.

I have also set up monitors and found that, if any "normal" file is found missing under this circumstance,

This sounds like a much bigger issue. Files do not accidentally vanish, unless there is some hard disk or (network) connection issue. And in these cases there will always be errors (unless the file system driver has bugs and fails silently).

and can be fully restored, while the corrupted ffs_db file CANNOT be restored.

What does this mean? Are you trying to restore deleted files with some tool?

[TerenceW]

Deleting files as a consequence of a corrupted ffs_db file cannot happen. FFS shows an error and defaults sync directions to copy to the other side.

I have also set up monitors and found that, if any "normal" file is found missing under this circumstance,

This sounds like a much bigger issue. Files do not accidentally vanish, unless there is some hard disk or (network) connection issue. And in these cases there will always be errors (unless the file system driver has bugs and fails silently).

and can be fully restored, while the corrupted ffs_db file CANNOT be restored.

What does this mean? Are you trying to restore deleted files with some tool?

1. I don't know, I am no professional, so I can only provide my guess or any clue I can. Now it seems that both 11.4 and 11.6 (no long-term verify yet) don't have this issue. I am posting here to see if there is really anything that needs to correct, fix, or pay attention to, especially for the future version. As a user, it never happened again since I change the version and avoid 11.5, I'm happy with these two versions.

2 & 3. Here I present to you how I repeat this phenomenon. The date of the issue is the same as when this thread was originally posted. The disk is a newly brought WD Elements SE 2TB (NTFS). I was able to repeat this bug 3 times, continuously, until I decided to give feedback, rollback to 11.4, and not risking my data.

i) On Computer A, I synced 1 new .docx file from the internal hard disk (DiskA) to this portable disk (PD), and the rule was "mirror". Now check PD, the file is present. Now eject PD, re-plug it, check, the file is present.

ii) On Computer B (with internal hard disk DiskB without this new file), plug PD, manually check the new file, it is present.

iii) Use the same .ffs_gui rule file as i), but with the opposite sync direction. Click compare, it prompts that the ffs_db file is corrupted. Click OK (close the warning window), click compare again, it tells that the two folders are the same. Now manually navigate to the folder, and find that this new file is both missing on PD and DiskB.

iv) Use the disk filesystem check tool in the right-click properties menu of Windows, click check for fs errors, and it says errors are found, and I can see a log. Open that log and it says ALL ffs_db files on both PD and DiskB are corrupted, but no other file types are corrupted. That missing new file is NOT shown in the log.

v) If you use the Windows chkdsk tool to fix this so-called fs error (prompts "success"), when you run FFS again, it will once more show the corruption message. If you delete all ffs_db file on PD and DiskB and let FFS re-create them, later when you re-open FFS, they will be corrupted AGAIN.

vi) Use a file recovery tool, like DiskGenius, you can find that the missing file is marked "deleted", and can be intactly recovered, but not the ffs_db files.

v) Computer A and B runs the same version of Windows 10 (latest release at that time, I don't remember the build number) and the same version of FFS (11.5). After I rollbacked to 11.4, this did not happen again.

vi) My path contains Chinese characters (incl. Chinese punctuations) and accented Latin letters.

vii) Other professional disk tool says no anomalies are found in my disk. Also, it is almost impossible that DiskB and PD failed at the same time and magically recovered after I switched the version. I admit that, to the best of my knowledge, this issue seems very very strange. But I'm not willing to risk my data again to repeat this experiment - sorry for that.

Again, thank you very much for this software, I really like it. I wish that my feedback could make it better.

[Zenju]

iv) Use the disk filesystem check tool in the right-click properties menu of Windows, click check for fs errors, and it says errors are found, and I can see a log. Open that log and it says ALL ffs_db files on both PD and DiskB are corrupted, but no other file types are corrupted.

It's not possible that such a file system corruption could be caused by FreeFileSync, not even by some bug.

Either your hard drive is broken or you are not properly unmounting your hard disk. I'd bet the latter is the cause for all the issues you observed.

[TerenceW]

iv) Use the disk filesystem check tool in the right-click properties menu of Windows, click check for fs errors, and it says errors are found, and I can see a log. Open that log and it says ALL ffs_db files on both PD and DiskB are corrupted, but no other file types are corrupted.

It's not possible that such a file system corruption could be caused by FreeFileSync, not even by some bug.

Either your hard drive is broken or you are not properly unmounting your hard disk. I'd bet the latter is the cause for all the issues you observed.

I am always very careful with the unmount process, especially for NTFS. FFS was the only thing I could think of. I've used FFS for a long time, and if it is an unmount issue, it wouldn't cause the file missing on DiskB for it is an internal drive. If the PD drive is broken, this could happen even if I rollback but it didn't. These two explanations cannot account for all the things I observed -- that's indeed quite strange.

Anyway, if nothing could be found from your and my ends, let's not focus on 11.4. If any future issues are observed, I'll also inform you.

BTW, anything I can do, like, catch a log and upload to the forum, when I encounter an issue in the future?

Thanks for your time.

[xCSxXenon]

Color me confused... That is borderline mind-blowing. I wonder if there is another software that sees 11.5 specifically and interrupts some process during read/write. I could see this issue happening if FFS crashed or was killed before finishing an operation