I also tried with scp and it is much faster, up to ~2MB/s
scp -v -P <port> -i <private-key-file> <source> <user@ipv4>:/
<source> 11% 44MB 1.7MB/s 03:21 ETA
To compare, I tried with a single file of ~500MB size.
Any reason for this?