Most efficient to Linux Server (SMB/SFTP/FTP)?

[durianwool]

Hi,

I have quite a big sync (800k to 1000k files, over 4TB) that I'd like to do daily as a mirror to a Linux server over a 1GBps LAN.

What is the most efficient protocol to use? I gathered SMB / Windows File Sharing is the least. But what about comparison between SFTP and FTP? I would initally think FTP since there's no ciphering overhead, but maybe there's some efficiency using SFTP?

I did the initial creation of the mirror over 13 hours, and moving forward it should only do the difference, which would be just gigs probably at each sync. However, there's still a need for FFS to I guess perform a local and remote listings, and then decide on the transfers affected. So which is the most efficient?

Thanks.

[durianwool]

OK - answering my own question. Surprisingly SFTP was faster in the scanning.
The 850k+ scan was done in 38secs with SFTP, but at 38secs with FTP it was only at the 500k files scanned mark. Guess I will use SFTP if only the scans are faster.

And tweaking further with setting below, I got the scan down about 10 secs!
Parallel file ops: 2
SFTP channels per connection: 2
Compression: off

I am not sure though when it comes to the actual file transfer having parallel is any better since I find usually the disks tends to start thrashing like when there a big movie file being transferred on 1 thread, while another thread is just making a lot of deletes and creates of smaller files.

Anyway - conclusion is SFTP is counter-intuitively more efficient.