Sometimes it seems to hang while copying a file

[Endian]

Hi,

I'm currently using ffs 10.20 on a project which includes a few million files per directory.
It works pretty good, but I have seen one problem.

I'm doing an update operation, i.e. I just add files to the new directory. No updating of already present files.
I'm having NTFS file systems for both source and target and run on Windows 10.
A couple of times I've noticed that it stops in the middle of copying, hanging on one specific file (according to the progress indication). It seems to update the remaining time, getting longer and longer but never seems to finish on the file I am copying (which is below a kB in size).
What I did then was to kill the process and restart it and then it works fine again.

There are no other processes writing to the source or target directories in this case.

[tcb123]

I have a similar problem. While synchronising a large number of files it sill stop copying on a file. The file count and amount copied stops increasing but it still appears to be copying. It is as if it has finished the file but does not realise it, so it carries on copying nothing. It has done this several times now on different files.

This is using version 10.22 on Windows 10 1909. I am copying file to a USB external hard disk.

[Endian]

I had a theory after seeing this on what could be causing it.

Maybe the file system's buffer is full so it flushes data to the disk? In that case it should be a temporary freeze, continuing after the flushing is done...

I haven't tried to verify it...

[tcb123]

I have been using FreeFileSync again today to copy a large number of files. I had it freeze three times. First time if froze for 4 minutes, second time for 6 and third time I gave up waiting after 10 minutes. It always froze when there was about 8 or 9 files left to copy out of a large number which makes it look like a bug. Restarting made it work OK.

I am using Windows Defender for AV.

FreeFileSync was still using about 4% CPU when it froze and task manager reported about 0.3 MB/s disk access from it.

[TommyTheTinkerer]

I have been testing putting exclusions in Windows defender for the folders you are accessing.
Too early to be totally sure, but seems to help so far. I have a few more jobs to run it on over the next week to see if it stands up.

[xCSxXenon]

I have seen in the past, sometimes the header in a file, or some data within the file, contains some corruption that causes a blockage in the chain. Whether it is in the program(FFS), the antivirus, the kernel, the storage controllers, the list goes on. The are so many factors that it hard to pinpoint the root cause. But, it is typically a service in the background or in deeper layers that has some bug.

[tcb123]

Turning windows defender real time scanning off makes not difference. I am still getting it freeze. Always when there are just a few files left to copy from a directory of very many files.

[Zenju]

Process Monitor is the first course of action to diagnose these types of problems.

[freetester23]

Same problem here.
If there are more then few files FFS hangs on Win10.

[Endian]

Sorry that I forgot to write down the process load numbers and allocated memory sizes for the thread, but that's a good point @Zenju. I'm sure I did check it though.

[tcb123]

I tried monitoring with Process monitor and it froze twice. From process monitor it looked like freefilesync was looping round but not frozen. I have sections of the logs here. The first time was a large file:
10:07:15.7298540 FreeFileSync_x64.exe 5752 QueryStandardInformationFile C:\Users\deb\sync.ffs_lock SUCCESS AllocationSize: 152, EndOfFile: 145, NumberOfLinks: 1, DeletePending: False, Directory: False
10:07:15.7298977 FreeFileSync_x64.exe 5752 WriteFile C:\Users\deb\sync.ffs_lock SUCCESS Offset: 145, Length: 1, Priority: Normal
10:07:15.7300662 FreeFileSync_x64.exe 5752 CloseFile C:\Users\deb\sync.ffs_lock SUCCESS
10:07:20.7430489 FreeFileSync_x64.exe 5752 CreateFile C:\Users\deb\sync.ffs_lock SUCCESS Desired Access: Generic Read/Write, Disposition: Open, Options: Synchronous IO Non-Alert, Open For Backup, Attributes: NT, ShareMode: Read, AllocationSize: n/a, OpenResult: Opened
10:07:20.7431591 FreeFileSync_x64.exe 5752 CreateFile C:\Program Files\FreeFileSync\Bin\FreeFileSync_x64.exe SUCCESS Desired Access: None 0x0, Disposition: Open, Options: , Attributes: N, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened
10:07:20.7432601 FreeFileSync_x64.exe 5752 QueryNameInformationFile C:\Program Files\FreeFileSync\Bin\FreeFileSync_x64.exe SUCCESS Name: \Program Files\FreeFileSync\Bin\FreeFileSync_x64.exe
10:07:20.7432843 FreeFileSync_x64.exe 5752 CloseFile C:\Program Files\FreeFileSync\Bin\FreeFileSync_x64.exe SUCCESS
10:07:20.7434631 FreeFileSync_x64.exe 5752 QueryNameInformationFile C:\Users\deb\sync.ffs_lock SUCCESS Name: \Users\deb\sync.ffs_lock

The second time was at the end of synchronising a user directory:

10:24:11.0795756 FreeFileSync_x64.exe 9312 QueryRemoteProtocolInformation F:\Users\Theo\RecycleBin~c737.ffs_tmp\AppData\Local\Google\Chrome\User Data\Default\Extensions\efaidnbmnnnibpcajpcglclefindmkaj\15.1.1.5_0\data\js INVALID PARAMETER
10:24:11.0795995 FreeFileSync_x64.exe 9312 QueryDirectory F:\Users\Theo\RecycleBin~c737.ffs_tmp\AppData\Local\Google\Chrome\User Data\Default\Extensions\efaidnbmnnnibpcajpcglclefindmkaj\15.1.1.5_0\data\js\frameUI.html SUCCESS Filter: frameUI.html, 1: frameUI.html, FileInformationClass: FileIdBothDirectoryInformation
10:24:11.0796403 FreeFileSync_x64.exe 9312 CloseFile F:\Users\Theo\RecycleBin~c737.ffs_tmp\AppData\Local\Google\Chrome\User Data\Default\Extensions\efaidnbmnnnibpcajpcglclefindmkaj\15.1.1.5_0\data\js SUCCESS
10:24:11.0797783 FreeFileSync_x64.exe 9312 CreateFile F:\Users\Theo\RecycleBin~c737.ffs_tmp\AppData\Local\Google\Chrome\User Data\Default\Extensions\efaidnbmnnnibpcajpcglclefindmkaj\15.1.1.5_0\data\js\frameUI.html SUCCESS Desired Access: Read Attributes, Delete, Read Control, Synchronize, Disposition: Open, Options: Synchronous IO Non-Alert, Open For Backup, Open Reparse Point, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened
10:24:11.0798578 FreeFileSync_x64.exe 9312 CreateFile C:\Program Files\FreeFileSync\Bin\FreeFileSync_x64.exe SUCCESS Desired Access: None 0x0, Disposition: Open, Options: , Attributes: N, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened
10:24:11.0799104 FreeFileSync_x64.exe 9312 QueryNameInformationFile C:\Program Files\FreeFileSync\Bin\FreeFileSync_x64.exe SUCCESS Name: \Program Files\FreeFileSync\Bin\FreeFileSync_x64.exe
10:24:11.0799223 FreeFileSync_x64.exe 9312 CloseFile C:\Program Files\FreeFileSync\Bin\FreeFileSync_x64.exe SUCCESS
10:24:11.0801069 FreeFileSync_x64.exe 9312 QueryNameInformationFile F:\Users\Theo\RecycleBin~c737.ffs_tmp\AppData\Local\Google\Chrome\User Data\Default\Extensions\efaidnbmnnnibpcajpcglclefindmkaj\15.1.1.5_0\data\js\frameUI.html BUFFER OVERFLOW Name: \Users\Theo\RecycleBin~c737.ffs_tmp\AppData\Local\Google\Chrome\User Data\Default\Extensions\
10:24:11.0801405 FreeFileSync_x64.exe 9312 QueryNameInformationFile F:\Users\Theo\RecycleBin~c737.ffs_tmp\AppData\Local\Google\Chrome\User Data\Default\Extensions\efaidnbmnnnibpcajpcglclefindmkaj\15.1.1.5_0\data\js\frameUI.html SUCCESS Name: \Users\Theo\RecycleBin~c737.ffs_tmp\AppData\Local\Google\Chrome\User Data\Default\Extensions\efaidnbmnnnibpcajpcglclefindmkaj\15.1.1.5_0\data\js\frameUI.html
10:24:11.0802170 FreeFileSync_x64.exe 9312 CreateFile C:\Program Files\FreeFileSync\Bin\FreeFileSync_x64.exe SUCCESS Desired Access: None 0x0, Disposition: Open, Options: , Attributes: N, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened
10:24:11.0802608 FreeFileSync_x64.exe 9312 QueryNameInformationFile C:\Program Files\FreeFileSync\Bin\FreeFileSync_x64.exe SUCCESS Name: \Program Files\FreeFileSync\Bin\FreeFileSync_x64.exe
10:24:11.0802716 FreeFileSync_x64.exe 9312 CloseFile C:\Program Files\FreeFileSync\Bin\FreeFileSync_x64.exe SUCCESS

[tcb123]

I think I have identified the cause of the problem I am having. When I back up a user directory on Windows 10 there are certain .exe files it is unable to copy (Error_cant_access_file) and I press ignore for each one. When it finishes copying there are 10 or 13 files left to copy corresponding to the ones ignored and it hangs trying to copy these non-existent files. I then have to use task manager to kill FreeFileSync.

This is happening with FreeFileSync 11.2 on the latest Windows 10.

[xCSxXenon]

What files are they? Can they be deleted?

[tcb123]

Some are exes under C:\Users\*\AppData\Local\Microsoft\WindowsApps, for example skype.exe.

I don't mind them not being copied. The problem is that FreeFileSync freezes up.

[xCSxXenon]

I would guess permissions based on the path... You can just exclude them from the sync by filtering them manually or right-click them after compare and exclude them that way.

[Endian]

It could be file permissions, but it could also be that the operating system is using the files.

Anyway, its not ok for a file sync program to freeze when trying to access them. I hope we can agree on that?

[tcb123]

It is not freezing when accessing the locked files. It is freezing when it has copied all the files, thinking there are still some to copy.