Search found 4 matches

by RedyAu
27 Oct 2023, 00:05
Forum: General Discussion
Topic: Scanning files takes hours over FTP - any ideas?
Replies: 5
Views: 1243

Re: Scanning files takes hours over FTP - any ideas?

Got the script working as far as I need it.
If anybody needs the same and finds this thread, here it is: https://github.com/RedyAu/ffs_hashgen/tree/main/bin

The .exe file is a Windows executable, beside the Dart source code. Might I add, that this is a very very sketchy way to download an ...
by RedyAu
26 Oct 2023, 17:35
Forum: General Discussion
Topic: Scanning files takes hours over FTP - any ideas?
Replies: 5
Views: 1243

Re: Scanning files takes hours over FTP - any ideas?

The "server" is just an old office PC running normal consumer Windows 10 😅
I have a FileZilla FTP over TLS server on it.
The fundamental bottleneck is that every single folder needs a separate listing command, there isn't much you can improve about that, I believe.

https://i.imgur.com/3WmGcWn.png ...
by RedyAu
25 Oct 2023, 22:03
Forum: General Discussion
Topic: Scanning files takes hours over FTP - any ideas?
Replies: 5
Views: 1243

Re: Scanning files takes hours over FTP - any ideas?

An archive flag doesn't solve the case where I remove files from my machine and I need them removed (or just moved to version history) on the remote server... But yes, a local "previous-state" database would help with this.
I guess this can be implemented quite easily in an outside script that does ...
by RedyAu
25 Oct 2023, 11:38
Forum: General Discussion
Topic: Scanning files takes hours over FTP - any ideas?
Replies: 5
Views: 1243

Scanning files takes hours over FTP - any ideas?

Hi!

I have a scheduled backup from my computer to my server over FTP.
Now that my files are synchronized, the actual incremental uploaded data is negligable. However, because millions of files are selected, it takes hours to scan the remote drive even with multiple threads.

The only way I see ...