Hello,
I'm syncing 2 million small files from a Windows 2008 server to a USB external drive with the Update method (FFS 5.10). The destination folder was empty, this is the first run and it took around 17 minutes to generate the file list. The file copy process is taking 4 days now and is still going.
Files are images of around 60KB plus thumbnails.
When the sync started, the transfer rate was 1MB/s and slowly degraded to 150 KB/s.
Any ideas ?
Thank you
Incredibly slow syncing 2 million small files
- Posts: 2
- Joined: 7 Jan 2013
- Posts: 141
- Joined: 10 Sep 2009
Try V5.11. I think it resolves that issue.
-
- Site Admin
- Posts: 7312
- Joined: 9 Dec 2007
USB storage devices, often FAT formatted, have very poor speed on random access, so this performance is probably "normal".
- Posts: 2
- Joined: 7 Jan 2013
Thanks for answering.
The USB drive is a 2TB LaCie formatted as NTFS. The server has 12GB RAM and 2 Xeon 6 Cores Processors.
The sync speed has worsen even more, its around 15 KB/s now. The FFS process is using 2GB Ram, is that normal ? The "Copy NTFS permissions" setting is enabled.
We are going to abort the sync, install FFS 5.11 and try syncing to a folder on the same server, just to rule out any problem with the USB drive. If that works fine then the next step is to do a standard file copy to the USB drive to see how long it takes, it's 88GB in total.
Best regards,
The USB drive is a 2TB LaCie formatted as NTFS. The server has 12GB RAM and 2 Xeon 6 Cores Processors.
The sync speed has worsen even more, its around 15 KB/s now. The FFS process is using 2GB Ram, is that normal ? The "Copy NTFS permissions" setting is enabled.
We are going to abort the sync, install FFS 5.11 and try syncing to a folder on the same server, just to rule out any problem with the USB drive. If that works fine then the next step is to do a standard file copy to the USB drive to see how long it takes, it's 88GB in total.
Best regards,
-
- Site Admin
- Posts: 7312
- Joined: 9 Dec 2007
srjones67 is right, it's probably:
viewtopic.php?t=1097
viewtopic.php?t=1097