Hi,
I'm trying to copy about 250,000 photos from one external veracrypt encrypted HDD toa another external veracrypt encrypted external HDD, both on different USB 3.1 connectors/Hubs.
In (donation) version 11.20 the copying starts out at about 35 MBye per second / about 10 elements per second, but drops off to about 3-5 Elements or 5-8 MByte per second after about 30s to 1minute. In version 11.20 increasing the number of parallel operations to 2 for both drives appears to have an effect, increasing but not quite doubling the speed.
I've searched this forum and found a thread that newer versions of FFS appear to be slower, and reinstalled donation version 11.12. Version 11.12 starts out at about 60-70 MByte per second / 20 elements per second, but equally drops off to the same speed after about 30s to 1m and appears to remain at that speed. Increasing the number of parallel operations doesn't appear to have an effect in this version.
Clearly, I'd prefer the 60-70MByte/s, so what can I do to keep up that speed? Is there a solution?
Thanks for any help!
Copy Speed dropping off dramatically after seconds
- Posts: 3
- Joined: 27 Apr 2022
-
- Posts: 4056
- Joined: 11 Jun 2019
You are likely filling up the cache on the HDD(s) and then it goes to the real-life speed of the drive. You will have to upgrade to much faster HDDs or just go full SSD.
- Posts: 3
- Joined: 27 Apr 2022
Thanks! But these are current seagate 2TB (expansion) / 4 TB (OneTouch) drives with alleged sustainable read/write speeds of >100 Mbytes/s, is it really realistic for them to be as slow as 5-10 MBytes/s sustainable copy speed for files ranging from 2-6 MBytes? I've been using FFS for years and it's never been that slow (at least I don't think so). Also this wouldn't explain the difference between 11.12 and 11.20?
- Posts: 1037
- Joined: 8 May 2006
I would run some tests, using Windows Explorer, using a file manager (& or dedicated copy utility) & see if you get consistent sustained high throughput or if they also drop off in speed.
If the latter, then it could be your drives.
(Different drives have differing characteristics & some drives, even labeled for a particular purpose - simply are not. [Like particular "NAS" drives using "SMR" technology. Western Digital Sued to Permanently Block SMR in NAS HDDs.])
If the latter, then it could be your drives.
(Different drives have differing characteristics & some drives, even labeled for a particular purpose - simply are not. [Like particular "NAS" drives using "SMR" technology. Western Digital Sued to Permanently Block SMR in NAS HDDs.])
- Posts: 3
- Joined: 27 Apr 2022
Thanks for the tip.
I've now run a couple of tests involving a set of 3650 photos (ranging from 1M mobile phone pics to 25M RAW files), all in all about 12 GB, and copied it around a bit.
One main result is: copying the set of files from src (2TB Seagate Expansion HDD, almoss full -> dest drive (4TB Seagate HDD, almost empty) (as in ffs) was at > 60MByte/s. Copying larger video files to and from SSD even resulted in sustained reads and writes on the hard drives of > 120 MByte/s.
The other main result: the "destination drive" (new Seagate One Touch 4TB, largely empty) was *much* slower at *reading* the smaller image files compared to the "source drive" (Seagate Expansion 2TB). The 2 TB drive read the images (1-6Mbytes) at around 60MBytes/s, the 4TB drive read the same files only at around 20MByte/s. Larger files (raws and videos) aren't read slower on the 4TB drives.
Writing speeds for *all* sizes was a little faster for the 4TB drive. Maybe b/c it was largely empty.
Since the only significantly reduced speed was for READING from the FFS destination drive, and there was no sustained drop off in speed in Windows Explorer, I'd cautiously say that this phenomenon is not related to the drop off. Is such a performance reasonable? Or is this a problem with the specific drive? Could this be related to the SMR feature?
I've also run FFS again after all the tests and the drop off is now not as pronounced as before which may indicate that there was an intervening variable present before. However, transfer speed in FFS still drops off about >20-40% in FFS compared to the file explorer experiments within the first minute and does not appear to recover until stopped and restarted (after a bit).
I've now run a couple of tests involving a set of 3650 photos (ranging from 1M mobile phone pics to 25M RAW files), all in all about 12 GB, and copied it around a bit.
One main result is: copying the set of files from src (2TB Seagate Expansion HDD, almoss full -> dest drive (4TB Seagate HDD, almost empty) (as in ffs) was at > 60MByte/s. Copying larger video files to and from SSD even resulted in sustained reads and writes on the hard drives of > 120 MByte/s.
The other main result: the "destination drive" (new Seagate One Touch 4TB, largely empty) was *much* slower at *reading* the smaller image files compared to the "source drive" (Seagate Expansion 2TB). The 2 TB drive read the images (1-6Mbytes) at around 60MBytes/s, the 4TB drive read the same files only at around 20MByte/s. Larger files (raws and videos) aren't read slower on the 4TB drives.
Writing speeds for *all* sizes was a little faster for the 4TB drive. Maybe b/c it was largely empty.
Since the only significantly reduced speed was for READING from the FFS destination drive, and there was no sustained drop off in speed in Windows Explorer, I'd cautiously say that this phenomenon is not related to the drop off. Is such a performance reasonable? Or is this a problem with the specific drive? Could this be related to the SMR feature?
I've also run FFS again after all the tests and the drop off is now not as pronounced as before which may indicate that there was an intervening variable present before. However, transfer speed in FFS still drops off about >20-40% in FFS compared to the file explorer experiments within the first minute and does not appear to recover until stopped and restarted (after a bit).
