Mirror syncs failed to make the target identical with the source

[N. T.]

Hi,

I'm using FreeFileSync version 13.5 on a Windows 10 PC.

I use "File time & size" for comparison, and "Mirror" for sync, to sync a folder on my PC to a folder in iCloud Drive.

I ran into the following 2 issues:

1. After each "successful" sync, there always remain, in the target folder, some "files that will be updated on the right side", and also some "files that will be deleted on the right side". In other words, the target folder remains an incomplete copy of the source folder.

2. After I make changes in a file in the source folder, if I don't change the file's name, FFS doesn't seem to recognize the changes, for it leaves the file in the target folder unupdated. I now have to resort to manually deleting the old file in the target folder, in order to force FFS to sync the changed file to the target folder.

Please advise how these issues may be resolved. Thanks!

[xCSxXenon]

Run a 'compare' and post a screenshot of the FFS window after it finishes.
You can attach files using the "Editor & Preview" below the text box

[N. T.]

Thanks. Here's the screenshot:

FFS 05.08 Log.jpg (168.13 KiB) Viewed 1788 times

[xCSxXenon]

We'll need the entire window, not a section

[N. T.]

Including folder & file names?

[N. T.]

Hi, may I cover up the folder & file names that are listed in the window? Thanks.

[N. T.]

We'll need the entire window, not a section

Hi, may I cover up the folder & file names shown in the window, or do you need to see those listed in the screenshot? Thanks.

[N. T.]

We'll need the entire window, not a section

[N. T.]

Hi, I just installed v13.6--it didn't solve my issues. Please help. Thanks!

[Plerry]

For files shown in your picture (some of the 78 files that FFS intends to update left=>right) it seems the size is identical left vs right. So, file size does not seem to be reason for a renewed sync.
You could check the file dates left-vs-right of some of these files. Perhaps iCloud modifies the file date of files in the iCloudDrive (sub)folder. If it does, the file time is different, and thus cause for sync action. And because you run a Mirror sync, it will be a left-to-right update of the right-side file.

As far as the 32 files FFS intends to delete in your ICloudDrive (sub)folder:
It seems those files (not shown in your picture) do exist in the iCloudDrive folder, but not (or no longer) in C:\Users. After you actually run the sync, FFS should have deleted those 32 files in the iCloudDrive (sub)folder; if not it will report the failure to do so in its log-file. You can check yourself right after the sync. Perhaps iCloud restores files that have been deleted.

But possibly you have a more fundamental problem:
Your left base location is "C:\Users" and your right base location
"C:\Users\ \iCloudDrive".
(I suppose the blanks in the latter base location are the result of some blanking for privacy reasons).
Because iCloudDrive is a (sub)subfolder of C:\Users, it is part of C:\Users, and hence there is a partial overlap between your left and right base location.If not dealt with properly, this can cause a perpetuum in sync actions. It will then ultimately create e.g. an iCloudDrive folder somewhere inside your iCloudDrive folder etc.
You can prevent overlaps between your left and right locations by using distinct, separate locations, or by adding an Exclude Filter rule to exclude, in your case, the (highest level) iCloudDrive folder.

[xCSxXenon]

Also, considering they are the same size, it is likely modification time that is changing. The 'source' files may have a timestamp unsupported in the destination (iCloud) and that will cause the timestamp to be set to the date/time of the sync operation. Also, iCloud may do some post-processing that changes the timestamp, thus a difference is created, and then FFS syncs it back to the initial state restarting the cycle.

Plerry does make a good point as well, having a parent directory syncing with/to one of its subdirectories presents some unique challenges that can take some complex/advanced tweaks to get working. What is your end goal here, as there might be a better solution? Why are you storing your data in the Users folder in the first place? This is an awful idea from a technical and organizational perspective, that's what the user library folders are for

[N. T.]

Thank you both. I'll look into the things you mentioned and then report back.

[N. T.]

Hello again.

First, a few questions related to my one-way sync issues:

Would any or all of the following lead to syncing errors in FFS?

1. Having folder/file names in another language than English.
2. Having many folder levels above the source folder (see below).
3. Having a folder in iCloud Drive as the destination folder for one-way syncing.
4. Having this date and time format (set in Windows 10)--e.g., "2024/05/17 周五 下午 1:30" (周五: Friday; 下午: PM).

5. Making small changes in a source file (e.g., replacing a word or two) which don't lead to file size changes.
6. Making changes in a source file on the same day, with editing times that are close together.


Regarding the locations of the source and destination folders, here are their full paths (I've "generalized" the names of the folders/subfolders):

Source folder: C:\Users\username\folder level 1\folder level 2\folder level 3\folder level 4\folder level 5\Poems.

Target folder: C:\Users\username\iCloudDrive\Poems.


Many thanks for helping me with the issues.

[Plerry]

1) Don't know. No experience with that
2) In theory no, but I have seen reports that a total path length over 255 characters caused problems. No clue if any of that is OS or file system related.
3) Don't know. No experience with that
4) That should affect just how date and time are displayed; not how date/time is stored.
5) Such changes are rare, but in any case should still result in a different time-stamp and thus cause a sync action.
6) Only if this results in less than 2 seconds (default) increase the the time stamp.
See FileTimeTolerance.
Or if the time-stamp difference matches exactly the "Ignore Time Shift" setting.

At least, you now seem to have eliminated the (partial) overlap between Source and Target location.

[N. T.]

Hi Plerry, thank you for answering my questions.

Now the question remains how, during a one-way sync, to persuade FFS--

1) To recognize changed files (even though the file names stay the same as before), so that it will sync them to the target folder in iCloud Drive; and

2) To delete from the target folder those files that are no longer in the source folder.


Thanks,

NT

[Plerry]

ad 1)
Was this ever a problem?
You wrote/showed that FFS wants to update files that (in your perception) had not changed.
I don't think you wrote that FFS does not sync files that had changed.

ad 2)
I expect this issue to be resolved by eliminating the overlap between Source and Target.
Can you confirm?

[N. T.]

Thank you again for helping me, Plerry.

Regarding "the overlap between Source and Target", well, it actually never existed: their full paths that I posted (with the folder names "generalized") stay the same as before--in my posted screenshot of the FFS window, I blanked out parts of their paths; sorry for the confusion. So it seems that the locations of the source and target folders didn't cause the incomplete mirror sync.


Regarding the syncing errors--I'm sorry if I didn't present them clearly--they are these 2:

1) FFS doesn't seem to recognize that a file in the source folder has been edited if I don't make changes in the file name as well--even though, obviously, the file's timestamp, and often, file size, have changed; as a result, the changed file is not copied from the source folder to the target folder during a mirror sync--i.e., after a "successful" sync, the older file is found in the target folder, instead the newly edited one. (This issue doesn't seem to occur if I make changes in the edited file's name; in that case, it gets recognized as a changed file, and copied to the target folder during a sync.)

2) When files in the source folder are replaced with newer versions, as well as modified titles, they remain in the target folder along with their newer versions after a mirror sync, resulting in more files in the target folder than in the source folder.

BTW, I've tried the mirror sync with the other 2 comparison methods--"File Content", and "File Size", and ran into the same 2 issues stated above.


Thanks,

NT

[N. T.]

One more issue--probably unrelated to the 2 syncing issues in question: FFS now & then reports an error, or gets stuck, because it cannot remove a file like RecycleBin~75e7.ffs_tmp from the target folder; I usually have to delete it manually to help it get unstuck.

[N. T.]

Also, during a mirror sync, FFS sometimes reports that it cannot delete a specific file. How could this be fixed?

Thanks.

[N. T.]

During the last few one-way syncs, I had to delete the “RecycleBin...tmp” file manually every time.

I got this error message once:

FFS Error Msg 2024-05-21 [10-09-25].jpg (69.82 KiB) Viewed 1384 times

[N. T.]

5/23
FFS was able to keep the source folder and the target folder identical this time; but at the bottom of the window, it stated that a difference remained: "281 files to be updated on the right side".


5/24
1. Had to manually delete 3 files from the target folder that FFS had been showing as "to be deleted", but hadn't been able to after 3 syncs.

2. FFS error message: "Unable to move "C:\Users\username\iCloudDrive\Poems\RecycleBin~53ec.ffs_tmp\2024 Poems\Filename.pdf" to the recycle bin. Error code 0x17b: The operation is not supported by the cloud sync provider.[IFileOperation::PerformOperations]"

3. RecycleBin~53ec.ffs_tmp was left in the target folder after a mirror sync was completed.

[N. T.]

5.24 Friday 18:24
FFS again failed to sync edited files to the target folder: the changed files have exactly the same sizes as their older counterparts, but their timestamps differ by about 5 hours (though on the same day).

[N. T.]

Hi FFS Support,

Could you provide some help with the above issues in one-way sync? Many thanks!

[N. T.]

Hello FreeFileSync Support,

Here again are the recurring issues that I've been having with mirror syncs:

1) FFS doesn't seem to recognize that a file in the source folder has been edited if I don't make changes in the file name as well--even though the file's timestamp has changed (though the file's size may not); as a result, the changed file is not copied from the source folder to the target folder during a mirror sync--i.e., after a "successful" sync, the older file is found in the target folder, instead of the newly edited one. (This issue doesn't seem to occur if I make changes in the edited file's name; in that case, it gets recognized as a changed file, and gets copied to the target folder during a sync.)

2) When files in the source folder are replaced with newer versions, as well as modified titles, they remain in the target folder along with their newer versions after a mirror sync--i.e., the sync has failed to make the target folder and the source folder identical.

3) During a one-way sync, FFS quite often reports an error, or gets stuck, because it cannot remove a file like "RecycleBin~75e7.ffs_tmp" from the target folder; I then have to delete it manually to help it get unstuck.

4) FFS sometimes reports that it cannot delete a specific file; the same file often stays undeletable by FFS through other syncs.


Thanks for any help you could give me on those issues.

[xCSxXenon]

This sounds like your configuration is completely broken/incorrect.
Click 'new' in the top-left and create the configuration from scratch

[N. T.]

I redid the configuration as follows (both the source and the target folders and their paths stay the same as before), and saved it as a batch job:

• Source/left side: C:\Users\Username\Foldername 1\Foldername 2\Foldername 3\Foldername 4\Foldername 5\Poems
• Target/right side: C:\Users\Username\iCloudDrive\Poems

• Compare - setting: File time and size
• Synchronize - setting: Mirror


Before I redid the configuration, I had changed a couple of words in 4 files, without changing their file names. I then ran a mirror sync, with these results: 2 of the 4 edited files got synced to the target folder; 2 didn't. The difference between the timestamps of the older files and their newer versions are 3--4 days.

Then I redid the configuration, and ran another mirror sync. This time FFS got stuck trying to delete "RecycleBin~37fd.ffs_tmp"; after about 50 minutes, I manually deleted the file. The results of this sync were the same as the last sync (i.e., before I redid the configuration): the 2 files that failed to get synced to the target folder in the previous sync remained unsynced in the latest sync.


Numbers in the logs stay the same before and after this latest mirror sync:

• Before: Completed Sucessfully:
上午 11:34:54 Comparison finished: 4,054 items found Time elapsed: 00:00:00
• After: Completed Sucessfully:
下午 12:46:17 Comparison finished: 4,054 items found Time elapsed: 00:00:00

Action view:
• Files that are equal: 1752
• Files that will be updated on the right side: 275
• 52.3 mb to copy

Difference view:
• Files that are newer on left: 2
• Files that are equal: 1752
• Files that are newer on left: 273
• 52.3 mb to copy


Thanks for looking into these stubborn mirror sync issues.

[N. T.]

Hello FFS support,

Could you suggest ways to deal with the mirror sync issues that I've reported, which have persisted?

I guess these issues must impact other users, too, though they may not be quite noticeable, since FFS reports the syncs as "successful" every time (except when it couldn't remove the recycle bin temp file). I hadn't noticed that the "successful" syncs were actually incomplete ones until I checked the contents of the files in the target folder.

Thanks for looking into those issues.

[N. T.]

Hi everyone,

Have any of you experienced those issues with mirror sync that I posted above? If so, have you found ways to solve them? Thanks.

I ran another mirror sync just now, with the source containing 2 newly edited files. FFS had the same erratic behavior as before: it copied one of them to the target, and ignored the other.

So, FFS doesn't seem to be able to consistently make the target into an exact copy of the source during a mirror sync, as it is supposed to do.

[N. T.]

Hi FFS support, have you been able to find solutions to the mirror sync's erratic behaviors? A mirror sync isn't really such if it cannot consistently make the target an exact copy of the source.