On non-SSD, non-NVME hard drives, does it pay off to increase thread count?

Discuss new features and functions
asdfasdf
Posts: 6
Joined: 7 Jun 2020

Post by asdfasdf

When comparing and copying files, does it pay off to increase thread count if you use external hard disk drives, which are not of SSD or NVME type?
xCSxXenon
Posts: 1126
Joined: 11 Jun 2019

Post by xCSxXenon

No, that won't help at all and may hurt performance even. Multi-thread is only useful when a connection/transfer is artificially limited, as then it just opens multiple connections at that speed. Anything local will be as fast as possible with a single thread, typically. It can't hurt to try, but mechanical drives are best when doing one operation at a time.
User avatar
Plerry
Posts: 1295
Joined: 22 Aug 2012

Post by Plerry

My experience is not in line with xCSxXenon's reply.
Both within my local (1Gb) network and over the Internet, I see a substantial increase in Compare and Sync speed when using multiple threads; the Sync part particularly when syncing a lot of relatively small files. (both for Local machine => NAS and NAS1 => NAS2)

In the Compare phase there is a lot of handshaking and relatively little data exchange; similarly when syncing small files. And that is where multiple threads can/will improve speed.

I would expect that for a Compare and Sync within your local machine there will be little benefit.
I have no experience syncing with USB HDs using multiple threads.
xCSxXenon
Posts: 1126
Joined: 11 Jun 2019

Post by xCSxXenon

That is almost exactly what I said though? In a properly configured network, it wouldn't benefit you at all to have multiple threads, but of course not all networks are the same.
User avatar
Plerry
Posts: 1295
Joined: 22 Aug 2012

Post by Plerry

> That is almost exactly what I said though?
I beg to disagree.
There is nothing in my connection or transfer that is artificially limited, and my network is properly configured and working properly (close to its 1Gb limit).
The slowness caused by the handshaking is protocol related (in my case SMB) and not network related.
xCSxXenon
Posts: 1126
Joined: 11 Jun 2019

Post by xCSxXenon

That's why I said not all networks work the same too. I have 10Gb LAN and once it hits the large amount of small files, it will drop to about 8.5Gb. If I use multiple threads, it drops as low as 7Gb transferring the same set of files. The difference is most likely the router's firmware, firewall rules, etc. I also have all antivirus completely disabled on the machines, even Microsoft Defender, which can help mitigate handshake overhead.