On non-SSD, non-NVME hard drives, does it pay off to increase thread count?
- Posts: 6
- Joined: 7 Jun 2020
When comparing and copying files, does it pay off to increase thread count if you use external hard disk drives, which are not of SSD or NVME type?
-
- Posts: 4056
- Joined: 11 Jun 2019
No, that won't help at all and may hurt performance even. Multi-thread is only useful when a connection/transfer is artificially limited, as then it just opens multiple connections at that speed. Anything local will be as fast as possible with a single thread, typically. It can't hurt to try, but mechanical drives are best when doing one operation at a time.
-
- Posts: 2450
- Joined: 22 Aug 2012
My experience is not in line with xCSxXenon's reply.
Both within my local (1Gb) network and over the Internet, I see a substantial increase in Compare and Sync speed when using multiple threads; the Sync part particularly when syncing a lot of relatively small files. (both for Local machine => NAS and NAS1 => NAS2)
In the Compare phase there is a lot of handshaking and relatively little data exchange; similarly when syncing small files. And that is where multiple threads can/will improve speed.
I would expect that for a Compare and Sync within your local machine there will be little benefit.
I have no experience syncing with USB HDs using multiple threads.
Both within my local (1Gb) network and over the Internet, I see a substantial increase in Compare and Sync speed when using multiple threads; the Sync part particularly when syncing a lot of relatively small files. (both for Local machine => NAS and NAS1 => NAS2)
In the Compare phase there is a lot of handshaking and relatively little data exchange; similarly when syncing small files. And that is where multiple threads can/will improve speed.
I would expect that for a Compare and Sync within your local machine there will be little benefit.
I have no experience syncing with USB HDs using multiple threads.
-
- Posts: 4056
- Joined: 11 Jun 2019
That is almost exactly what I said though? In a properly configured network, it wouldn't benefit you at all to have multiple threads, but of course not all networks are the same.
-
- Posts: 2450
- Joined: 22 Aug 2012
> That is almost exactly what I said though?
I beg to disagree.
There is nothing in my connection or transfer that is artificially limited, and my network is properly configured and working properly (close to its 1Gb limit).
The slowness caused by the handshaking is protocol related (in my case SMB) and not network related.
I beg to disagree.
There is nothing in my connection or transfer that is artificially limited, and my network is properly configured and working properly (close to its 1Gb limit).
The slowness caused by the handshaking is protocol related (in my case SMB) and not network related.
-
- Posts: 4056
- Joined: 11 Jun 2019
That's why I said not all networks work the same too. I have 10Gb LAN and once it hits the large amount of small files, it will drop to about 8.5Gb. If I use multiple threads, it drops as low as 7Gb transferring the same set of files. The difference is most likely the router's firmware, firewall rules, etc. I also have all antivirus completely disabled on the machines, even Microsoft Defender, which can help mitigate handshake overhead.
