Huge file FTP transfer

[leopondolo]

Hi everyone,
I am experiencing troubles in uploading a 100+ GB file to an FTP server.
If upload a smaller file (<20GB) all is ok and the FTP log (filezilla server side) is something like this:

(000357)20/01/2020 14.52.07 - <my_username> (<public_ip_of_ffsync_client>)> EPSV
(000357)20/01/2020 14.52.07 - <my_username> (<public_ip_of_ffsync_client>)> 229 Entering Extended Passive Mode (|||53930|)
(000357)20/01/2020 14.52.07 - <my_username> (<public_ip_of_ffsync_client>)> STOR /TS-DB~34c6.ffs_tmp
(000357)20/01/2020 14.52.07 - <my_username> (<public_ip_of_ffsync_client>)> 150 Opening data channel for file upload to server of "/TS-DB~34c6.ffs_tmp"
(000357)20/01/2020 14.52.07 - <my_username> (<public_ip_of_ffsync_client>)> 226 Successfully transferred "/TS-DB~34c6.ffs_tmp"
(000357)20/01/2020 14.52.07 - <my_username> (<public_ip_of_ffsync_client>)> MFMT 20200119100038 /TS-DB~34c6.ffs_tmp
(000357)20/01/2020 14.52.07 - <my_username> (<public_ip_of_ffsync_client>)> 213 modify=20200119100038; /TS-DB~34c6.ffs_tmp
(000357)20/01/2020 14.52.07 - <my_username> (<public_ip_of_ffsync_client>)> RNFR /TS-DB~34c6.ffs_tmp
(000357)20/01/2020 14.52.07 - <my_username> (<public_ip_of_ffsync_client>)> 350 File exists, ready for destination name.
(000357)20/01/2020 14.52.07 - <my_username> (<public_ip_of_ffsync_client>)> RNTO /TS-DB.vbm
(000357)20/01/2020 14.52.07 - <my_username> (<public_ip_of_ffsync_client>)> 250 file renamed successful


When the file is huge the phase MFMT-RNFR-RNTO is missing. The client simply disconnects and after 3 minutes it logs in back and delete the .tmp file, like this:

(000361)20/01/2020 23.14.46 - <my_username> (<public_ip_of_ffsync_client>)> EPSV
(000361)20/01/2020 23.14.46 - <my_username> (<public_ip_of_ffsync_client>)> 229 Entering Extended Passive Mode (|||57958|)
(000361)20/01/2020 23.14.46 - <my_username> (<public_ip_of_ffsync_client>)> STOR /TS-DBD2020-01-19T104756_D634~17e2.ffs_tmp
(000361)20/01/2020 23.14.46 - <my_username> (<public_ip_of_ffsync_client>)> 150 Opening data channel for file upload to server of "/TS-DBD2020-01-19T104756_D634~17e2.ffs_tmp"
(000361)21/01/2020 04.39.37 - <my_username> (<public_ip_of_ffsync_client>)> 226 Successfully transferred "/TS-DBD2020-01-19T104756_D634~17e2.ffs_tmp"
(000361)21/01/2020 04.39.56 - <my_username> (<public_ip_of_ffsync_client>)> disconnected.
(000363)21/01/2020 04.42.58 - (not logged in) (<public_ip_of_ffsync_client>)> Connected on port 21, sending welcome message...
(000363)21/01/2020 04.42.58 - (not logged in) (<public_ip_of_ffsync_client>)> 220-FileZilla Server 0.9.60 beta
(000363)21/01/2020 04.42.58 - (not logged in) (<public_ip_of_ffsync_client>)> 220-written by Tim Kosse (<email of filezilla creator>)
(000363)21/01/2020 04.42.58 - (not logged in) (<public_ip_of_ffsync_client>)> 220 Please visit **********/
(000363)21/01/2020 04.42.58 - (not logged in) (<public_ip_of_ffsync_client>)> USER <my_username>
(000363)21/01/2020 04.42.58 - (not logged in) (<public_ip_of_ffsync_client>)> 331 Password required for <my_username>
(000363)21/01/2020 04.42.58 - (not logged in) (<public_ip_of_ffsync_client>)> PASS ******
(000363)21/01/2020 04.42.58 - <my_username> (<public_ip_of_ffsync_client>)> 230 Logged on
(000363)21/01/2020 04.42.58 - <my_username> (<public_ip_of_ffsync_client>)> PWD
(000363)21/01/2020 04.42.58 - <my_username> (<public_ip_of_ffsync_client>)> 257 "/" is current directory.
(000363)21/01/2020 04.42.58 - <my_username> (<public_ip_of_ffsync_client>)> CLNT FreeFileSync
(000363)21/01/2020 04.42.58 - <my_username> (<public_ip_of_ffsync_client>)> 200 Don't care
(000363)21/01/2020 04.42.58 - <my_username> (<public_ip_of_ffsync_client>)> OPTS UTF8 ON
(000363)21/01/2020 04.42.58 - <my_username> (<public_ip_of_ffsync_client>)> 202 UTF8 mode is always enabled. No need to send this command.
(000363)21/01/2020 04.42.58 - <my_username> (<public_ip_of_ffsync_client>)> DELE /TS-DBD2020-01-19T104756_D634~17e2.ffs_tmp
(000363)21/01/2020 04.42.59 - <my_username> (<public_ip_of_ffsync_client>)> 250 File deleted successfully
(000363)21/01/2020 04.43.20 - <my_username> (<public_ip_of_ffsync_client>)> QUIT
(000363)21/01/2020 04.43.20 - <my_username> (<public_ip_of_ffsync_client>)> 221 Goodbye
(000363)21/01/2020 04.43.20 - <my_username> (<public_ip_of_ffsync_client>)> disconnected.

Any idea on how to make the transfer work ?
Thank you

[leopondolo]

No idea at all ???
Developers ??

[Zenju]

What error message is FreeFileSync showing?

[leopondolo]

This is what FFSYNC log says:
(translated in english)
16:33:40 Error Cannot write the file "ftp://<my_ftp_site_address>/TS-DBD2020-01-19T104756_D634~b5c8.ffs_tmp".
CURLE_OPERATION_TIMEDOUT: FTP response timeout
150 Opening data channel for file upload to server of "/TS-DBD2020-01-19T104756_D634~b5c8.ffs_tmp" [curl_easy_perform]
16:33:40 Error Stop

[Zenju]

I don't think this time out is caused by anything set up by FFS (assuming that the user-configurable time out is large enough, which FFS will pass on via CURLOPT_FTP_RESPONSE_TIMEOUT). The project lead of libcurl has this to say:
https://curl-library.cool.haxx.narkive.com/97FKNitM/curle-operation-timedout-when-ftp-upload-with-large-files#post2

[Zenju]

But maybe we can work around. Here's a version enabling CURLOPT_TCP_KEEPALIVE. Does it fix the time out problem?
https://www.mediafire.com/file/jhjz5s4ymufc2qk/FreeFileSync_10.20_%5BBeta%5D_Windows_Setup.exe

[leopondolo]

I will test it tonight, Zenju. Thank you very much for your support !!! :-)

[leopondolo]

It worked like a charm, Zenju. Thanks a lot for your support. FFSYNC is top !!!!

[douzaine]

Zenju, I may be having a similar problem.

I tried to download the Beta version enabling CURLOPT_TCP_KEEPALIVE in your post above, but it's not on Mediafire anymore.

Could you please provide me with another link?

[xCSxXenon]

douzaine, as of version 10.20, that feature is implemented in the stable release. Download any version equal to or newer and you are set