Performance Enhncement

[RelayUK]

Performance enhancement thought...

Copy 100K small files (total 10GB) = x
Copy 1 large file (total 10GB) = x / 10

i.e. small files copy 10 times slower.

One thought would be to take the list of files to copy, sort them by size, and have two copy threads, one at each end of the list. so one thread copies large files while the other copied small files.

In a general scenario this will ensure a much higher copy speed. For only small files, you get at least a 2x performance increase.

[Zenju]

Only if latency is the bottleneck.

[RelayUK]

The I/O required to create a file and close it, seek to a new location for the next file is greater than the time to write a small file, especially on NAS, USB, and spindle disks. Even SSDs suffer from this.