ERROR_FILE_EXISTS [CopyFileEx] on Samba share

[pkiff]

Are you able to reproduce ERROR_FILE_EXISTS that is leaving behind a zero-byte file with FFS 11.14 for a specific file?

In my case, no I can't - at least not until the error pops up again. When (if) it does I will pay closer attention and do a couple tests. Currently, I'm testing to see if simply pausing OneDrive synchronization before syncing with FFS will eliminate the error. Synced once now with no issues, but I only synced a dozen files. The issue may require larger sync sets to show up.

[pkiff]

I'm still using FFS 11.14. I ran into this issue again during a sync of about 20 files. One new file to be synced failed with a "ERROR_FILE_EXISTS: The file exists. [CopyFileEx]" error. The error states the correct source file name and path but and shows the correct Buffalo Linksation NAS network folder (starting with "\\LS-220\...." but the destination filename is a temp file, with shortened name like this: "FileNameBeginsCorrectlyThenTilda~4dde.ffs_tmp" file. After the failed sync, there is then a 0-byte .ffs_tmp file left in the folder. That file prevents any further syncing of the original file. But even after deleting that file, I could not sync the original file today. What I had to do was MOVE the original file to the parent folder, then sync everything again, and then it worked. After that, I moved the culprit file back to its original location, and it synced again correctly. So the same file in the same folder failed to sync the first time, but then was able to sync later in the same place.

Initially, I tried this with my corporate OneDrive running in the background as well. I thought that might be the culprit, but even after turning it off, and deleting that ffs_tmp file, in this case, the sync wouldn't proceed until I moved it to another folder.

I don't have any further theories at the moment, but am happy to try 11.15 beta if you are looking for further testing. Or I'll just update to 11.15 when it is officially released and see if that solves the issue.

[Marek-M]

Next week with backup and no NAS synchronize problems. For me issue is fixed in FreeFileSync Beta 11.15 2021-10-11

Sounds promising! I can't release the code in the beta in the current form. First need to find out, which setting did the trick:

Are you able to reproduce ERROR_FILE_EXISTS that is leaving behind a zero-byte file with FFS 11.14 for a specific file?

If yes, let's do a few more tests:

1. Are able to copy with this version? https://www.mediafire.com/file/g0dh2z06edwysxd/FreeFileSync_11.15_%255BBeta%255D_Windows_Setup%25287%2529.exe
2. And with this? https://www.mediafire.com/file/6au4qqxd5ej85e9/FreeFileSync_11.15_%255BBeta%255D_Windows_Setup%25288%2529.exe
3. This one? https://www.mediafire.com/file/je3629022tumg9d/FreeFileSync_11.15_%255BBeta%255D_Windows_Setup%25289%2529.exe

I will try to do these tests next Sunday - this is my backup day :) Thank you for your patience.

I just finished syncing files. I made it using the application from point 1:

1. Are able to copy with this version? Https://www.mediafire.com/file/g0dh2z06edwysxd/FreeFileSync_11.15_%255BBeta%255D_Windows_Setup%25287%2529.exe

All files copied correctly and further application tests from points 2 and 3 were not possible.
Processed items: 302 (664 MB)

[Zenju]

but am happy to try 11.15 beta if you are looking for further testing. Or I'll just update to 11.15 when it is officially released and see if that solves the issue.

When you encounter such a reproducible case again, a test with version 2, and if it fails, version 1 (in that order) would help a lot.

[pkiff]

When you encounter such a reproducible case again, a test with version 2, and if it fails, version 1 (in that order) would help a lot.

I ran into the error today with a single file failing out of about 200 changed files that were synced during a manual sync of about 27,000 files using v1.14. Unfortunately, the same file caused errors with both the version 2 and version 1 from those Mediafire download locations in that other post. The error is slightly different as it reads "ERROR_FILE_EXISTS: The file exists. [CreateFile]" instead of [CopyFileEx]. And instead of starting by saying "Error Cannot copy file" it says "Error Cannot write file". But other than that the behaviour for me was the same in all versions.

I was only able to fix it after moving the culprit file and then re-syncing. After which, I could put it back in the same place it was originally, and sync again and all was good.

In this case, my first attempt at syncing was done with OneDrive running. Eventually I tried synching with OneDrive paused. But I still wonder if it would have synced correctly the very first time if I had paused OneDrive before starting the very first sync. I intend to try to remember to pause OneDrive each time from now on before running FFS and see if the error appears again.

The culprit file in this case was an Excel file with an .xlsx extension and a long file name inside a long sequence of subfolders. The total characters used by filename + subfolders was 205 characters.

It seems possible to me that my issues may be unique and not the same as the OP. It does occur on a Samba share, but there are lots of other variables. The Samba share on my local network has restricted user access. I am syncing from a local folder on an encrypted C drive that is in turn synced via OneDrive to a different remote network Microsoft location that requires a corporate login to access. The Samba share is on a virtual drive on a Buffalo Linkstation that is interpreting protocols for access from Windows machines on the network. There seem to be lots of things that could go wrong all along those paths. And I don't have full control over my corporate machine, so there may be anti-virus and/or spyware managed by my corporate admin that is interfering with things that I am unaware of.

As a reminder, I personally haven't had this issue when syncing from my personal machine using the same version of FFS (donation version, though), connected to the exact same Samba network share using similar protocols. It's only my corporate machine where I'm getting stray issues once in a while.

But like I say, I'm going to try to remember to turn off OneDrive syncing my corporate machine from now on whenever I run FFS and I'll see if that solves the problem.

Thanks for your ongoing efforts. FreeFileSync is truly awesome software that I am very grateful for.

[Marek-M]

but am happy to try 11.15 beta if you are looking for further testing. Or I'll just update to 11.15 when it is officially released and see if that solves the issue.

When you encounter such a reproducible case again, a test with version 2, and if it fails, version 1 (in that order) would help a lot.

Today's file backup was relatively small:
Processed items: 46 (74,9 MB)

Using version 2 of the program I did not get any error related to the message "ERROR_FILE_EXISTS". For now, I am planning more backups with this version 2 of the program. When these errors appear again, I will notify you as soon as possible.

[Zenju]

Best course of action would be to use 11.14 until a (hopefully reproducible) ERROR_FILE_EXISTS occurs, then switch to one of the betas to see if they can actually solve it.

[Zenju]

The error is slightly different as it reads "ERROR_FILE_EXISTS: The file exists. [CreateFile]" instead of [CopyFileEx]. And instead of starting by saying "Error Cannot copy file" it says "Error Cannot write file". But other than that the behaviour for me was the same in all versions.

It seems to me this is a similar case as OP, but not the same. It's likely the OP case plus adding OneDrive to the mix. The faulty CopyFileEx call is detected correctly by both betas, but it seems the cleanup is not working. Probably because OneDrive is locking the file.

[Zenju]

Probably because OneDrive is locking the file.

Just found a bug in the beta, which fails to properly delete the bogus 0-size file.

=> Back to the drawing board. I'll be preparing two new betas.

[pkiff]

Just found a bug in the beta, which fails to properly delete the bogus 0-size file.

=> Back to the drawing board. I'll be preparing two new betas.

I'm happy to test a beta version when/if I encounter an issue again - though your analysis may be correct - and issues with OneDrive are probably very annoying to work around. Thanks again.

[Zenju]

Okay let's go for another round:

• First make sure the NTFS extended attribute error is in fact occuring and reproducible by using FFS 11.14. Otherwise we won't know if the betas fixed something or not.

• If it is, switch to the following beta (version 1) and see if this fixed the issue, or post the error message if not. https://www.mediafire.com/file/r32k6b0bcf54nqt/FreeFileSync_11.15_%255BBeta%255D_Windows_Setup%252812%2529.exe

• If beta version 1 doesn't fix it, maybe beta version 2 does?
https://www.mediafire.com/file/ecugo9uio31z71x/FreeFileSync_11.15_%255BBeta%255D_Windows_Setup%252813%2529.exe

[Marek-M]

I will try to do tests this weekend.

Does it matter for testing whether "FFS 11.14" is standard or "Donation_Edition"?

[Zenju]

I will try to do tests this weekend.

Does it matter for testing whether "FFS 11.14" is standard or "Donation_Edition"?

No, doesn't matter.

[Marek-M]

I reinstalled FreeFileSync 1.14 and have just backed up to the NAS. Unfortunately (or fortunately :)) no file error occurred.

Processed items: 52 (314 MB)

In a week I can make another try.

[john78]

When mirroring some folders I get the same message every time. Is there no one (from FFS ?) who knows the solution (software bug ?) ? Do I have to look for another app to secure my folders ?

[Zenju]

When mirroring some folders I get the same message every time. Is there no one (from FFS ?) who knows the solution (software bug ?) ? Do I have to look for another app to secure my folders ?

Maybe you can help out: viewtopic.php?t=7898&start=30#p31858

[watrworld]

When mirroring some folders I get the same message every time. Is there no one (from FFS ?) who knows the solution (software bug ?) ? Do I have to look for another app to secure my folders ?

I now use SyncFolders, also free, works for me until FFS is fixed.

[Zenju]

I now use SyncFolders, also free, works for me until FFS is fixed.

Without someone doing the testing, it can't be fixed.

[watrworld]

Without someone doing the testing, it can't be fixed.

I could test if I am provided with a Portable version.

[Zenju]

The betas aren't portable, unfortunately.

[Marek-M]

A week ago and a moment ago I made a backup with "FFS 1.14" on the NAS (Samba share) and there were no "CopyFileEx" errors. I don't know what happened but "FFS" fixed itself or one of the Windows updates changed something.

The program said that version 1.15 is now available. Due to the fact that for two weeks I have not been able to get errors in synchronization in version 1.14, I think I can update the application. @Zenju, you don't need to test version 1.14, which works fine?

[john78]

Hello, I have installed version 11.15 beta. I have already mirrored three directories and so far no Copyfile errors have been reported.
Yours sincerely,

[dskouras]

In my case I've had this problem for while and assumed it was an issue with my setup. I just now realized there's a problem with the Application.

I am using 11.15 donation version and have the issue on all my syncs from NTFS drives to my synology NAS.

I've tried running the software under windows 10 and Debian 11 and I get the problem in both cases (actually in Debian I get a few more errors that I can't be bothered to debug, so I just click ignore.

Is there any advice on how to fix this?I really need the sync processes to run...
Happy to send any logs that might be useful

[Zenju]

In my case I've had this problem for while and assumed it was an issue with my setup.

Is it reproducible? Are you able to simplify your sync to a single file that exhibits the ERROR_FILE_EXISTS error?

I just now realized there's a problem with the Application.

Well the problem lies not with FFS, but the software could implement a workaround to what appears to be a bug in Microsoft's CopyFileEx().

I've tried running the software under windows 10 and Debian 11

Interesting, whatever errors you're seing in Debian, it's almost certainly something else than what is discussed in this thread, which is a Windows-specific issue.

[dskouras]

Hi and thank you for replying!

In my case I've had this problem for while and assumed it was an issue with my setup.

Is it reproducible? Are you able to simplify your sync to a single file that exhibits the ERROR_FILE_EXISTS error?

My sync's all generally sync mirror the contents of folders and drives on my desktop to my synology NAS.

I tested several sync pairs, from a simple one with one file to more complex ones with hundreds of files including documents, photos and software. All ran fine with no (significant errors)

However two particular sync pairs fail with this error - one syncing my home videos and another with several hundred archives (zip/rar etc). There I got several hundred "ERROR_FILE_EXISTS: The file exists. [CopyFileEx]" instances - basically one for every file to be copied, so no files were copied.

I have no idea why these two fail, but one thing they have in common is the size of the files - the smallest one is about 50MB and the largest a few gigabytes.

I tried creating various combinations of sync settings for these files but all failed.

I also got a new error on one of them:
"Cannot open directory "F:\Recovery".
ERROR_ACCESS_DENIED: Access is denied. [DirReaderPlus]"
Its confusing because there is no file or folder called Recovery on this drive, not even a hidden one.

I hope this helps a bit, as I said happy to provide logs etc if useful,

I just now realized there's a problem with the Application.

Well the problem lies not with FFS, but the software could implement a workaround to what appears to be a bug in Microsoft's CopyFileEx().

Understood, thank you for clarifying.

I've tried running the software under windows 10 and Debian 11

Interesting, whatever errors you're seing in Debian, it's almost certainly something else than what is discussed in this thread, which is a Windows-specific issue.

I'm currently on windows 10, but I will reboot to Debian (it's a dual boot system), run it again and report back. I should mention to be precise that I'm running Debian testing, so in fact not the most solid environment. Not sure if that might play a part.

[dskouras]

OK, I can confirm that the issue does not occur in Debian. The error I got was another one about another one about not having access to sync.ffs_tmp

Syncs work as expected in Debian.

[Zenju]

There I got several hundred "ERROR_FILE_EXISTS: The file exists. [CopyFileEx]" instances

Sounds good (as a test case). But first simplify the scenario by setting an include file filter so that only a single file that fails copying with ERROR_FILE_EXISTS will be synced.
I'll also need a Process Monitor log to verify that FFS's workaround is in fact doing the right thing.

So the procedure then is:

1. Start Process Monitor and begin logging: https://freefilesync.org/faq.php#trace
2. Run the sync (with a single failing file) by using this FFS beta: https://www.mediafire.com/file/nrwos2l1lk7byf8/FreeFileSync_11.16_%255BBeta1%255D_Windows_Setup.exe
3. Hopefully this time the sync worked, then send me the pml file via email.

(4.) Retry the steps above using beta version 2, which uses a slightly different workaround: https://www.mediafire.com/file/1kbuof4gnn5vqsl/FreeFileSync_11.16_%255BBeta2%255D_Windows_Setup.exe

"Cannot open directory "F:\Recovery".
ERROR_ACCESS_DENIED: Access is denied. [DirReaderPlus]"
Its confusing because there is no file or folder called Recovery on this drive, not even a hidden one.

This is an unrelated error. "F:\Recovery" does in fact exist and is internal to Windows. It's not only marked "hidden", but also "system".

[Chribaa]

my severe issues (see my post from Jul 24, 2021) have been successfully resolved with the first beta, that Zenju had posted on Dec. 27.
@Zenju: thanks so much for this. Now FFS becomes a useful tool for me!

[Zenju]

my severe issues (see my post from Jul 24, 2021) have been successfully resolved with the first beta, that Zenju had posted on Dec. 27.
@Zenju: thanks so much for this. Now FFS becomes a useful tool for me!

For the beta to become official I still need the Process Monitor trace for verification, and a second trace for beta version 2.

[Chribaa]

@Zenju, as per my mail it is difficult to provide a full PML due to the laptop being corporate. If you can point out, what specifically you need, I could send screenshots of these parts.