Using the option to compare file contents yields false comparison errors

[hsehestedt]

I recently migrated a large amount of data from one computer to another using FreeFileSync. Out of an abundance of caution, I decided to perform a comparison of the data using the option to compare file contents. Out of 3,000+ files FreeFileSync identified 27 files as being different.

I then re-synced those files.

However, I also sync the same source to another location so I ran a comparison in the same manner. This time only one file was found to be different. However, this time, rather than simply syncing the source over to the destination, I performed a CRC check of both the source and destination files. Here is what I found:

The MD5 hash for both files is identical.
The SHA256 hash for both files is identical.
The date created is the same.
The Modified data is the same.
The file size is the same.

I have preserved a copy of both the source and destination files just in case any further analysis should be needed.

I compared the files again with FreeFileSync and it still claimed the files to be different, so at least I know that it was not just a one-time glitch.

I then synced the original source file to the destination using FreeFileSync, then ran another comparison. Now the comparison passes.

Would it be possible for you to tell me how FreeFileSync determines a difference in files?

NOTE: In the first case where FFS showed 27 files to be different, I know that at least some of those files were legitimately different. I know this because the one sample file that I looked at reported a CRC error when I tried to unzip that file on the destination. On the source, the same file was just fine. Unfortunately, that was the only file I looked at before re-syncing.

However, I have further opportunity for looking into this. I have another 6TB of data in different folders of the same source and destination that I have not yet run a comparison against. I'm about to begin that process now.

[xCSxXenon]

https://freefilesync.org/manual.php?topic=comparison-settings
bit-by-bit by the sounds of the above

[hsehestedt]

New information on the above issue:

I have compared 2 folders that FreeFileSync has replicated. These folders are on two different PCs so the comparison is happening over the network. Each folder holds about 68GB of data with file sizes ranging from 544KB to about 5GB.

I ran this comparison 3 times in a row and each time FreeFileSync tells me that a different number of files are different.

Run #1: FreeFileSync claims that 12 files are different
Run #2: 8 files are different
Run #3: 14 files are different

In the meantime, I have also compared about 6TB of data on the same 2nd computer but NOT over the network. This comparison was all between USB connected HDDs. The result: Not one sinle file was reported to be different.

Clearly, there is an issue comparing files over the network. Any thoughts on what might cause this?

[xCSxXenon]

A bad network connection certainly? Sounds like packet loss. Hard to say/guess, networking is complicated.
Reducing 'hops' between computers is a start. What does the path between the two look like, hardware-wise? (Switches, router, wifi/ethernet, etc.)

[therube]

I also sync the same source to another location

Also over the network?

This time only one file was found to be different.

I performed a CRC check of both the source and destination files. Here is what I found:

The MD5 hash for both files is identical.
The SHA256 hash for both files is identical.

So you're saying FFS reported one file that did not Content compare.

But then you manually ran a hash check on that one source file & on that destination file, & they did compare.

So one time it failed (FFS) & one time it worked (manual hash) [on the same two files].

So it does sure seem the files are identical but not always ;-).
Other then network, potentially RAM, but you'd think less likely (as other local tests you did were successful).

[hsehestedt]

Update and final resolution...

The issue does indeed seem to have been a network issue. Without going into all the sordid details, I simplified the network configuration on both the source and destination and gave my hypervisor a separate network interface rather than share an interface with the physical system, and things are much better now.

I transferred a few hundred GB between source and destination using FFS, then performed a file compare, again, using FFS. All was good.

BTW, I confirmed that there were actually some real binary differences in some of the files reported as being different before, while others were simply false indications. I'm very glad that I discovered this now before it turned into a major issue :-)

Very thankful to FFS for providing the ability to do a real file comparison! Absolutely indispensable tool for me!