Runs much slower after batch scheduling

Get help for specific problems
Posts: 1
Joined: 31 Jan 2024

mrxo

I'm syncing to a remote SFTP location, I've also tried a remove SMB location. When I first create a folder pair the sync happens quickly (20-30min), however when I schedule it via scheduled task it gets slower and slower each run.

I'm using the donation edition, 10 parallel threads on the local side and 20 on the remote side, 10 SFTP channels per session. I've already read on the forums that 1 parallel operation on the local side goes just as fast but in my configuration 10 goes significantly faster. For the remote side 20 seems to be a good spot for speed.

When the job is going quickly, the local side of the scanning process takes around 1 minute, running at 250k or so items/s. Remote runs at ~15000 items/s. I've ran a couple scheduled task batch runs and the local side is now doing less than 2000 items/s, making the scanning part last hours instead of the 20 minutes that the whole job takes when running it manually. Sometimes, the scanning part will hang and stop processing for awhile (usually 5 minutes) and then continue. On the SFTP side I'm using openssh 8.9p1 and I've tried increasing the maxsessions and maxstartups. I have set TCPKeepAlive to yes. I see no errors in the SSH logs. When it restarts from hanging I see events like this for each thread and then it starts again.

Jan 31 15:49:11 ip-10-200-247-124 systemd[1]: session-649.scope: Deactivated successfully.
Jan 31 15:49:11 ip-10-200-247-124 systemd[1]: session-649.scope: Consumed 14.943s CPU time.
Jan 31 15:49:11 ip-10-200-247-124 systemd[1]: Started Session 659 of User ubuntu.


This is my command line for the scheduled task:
"C:\Program Files\FreeFileSync\FreeFileSync.exe" "D:\Temp\SFTP2.ffs_batch"
I have it running only when user is logged in just so I can monitor the status from the gui which doesn't pop up when I run it whether the user is logged in or not. I've tried running it with and without highest privileges.

I've played with the settings a lot and I can't seem to find a reason for this to run slowly. Can someone help me determine what the reason is for this speed differential? I'd like to run this job once per hour.