Fast on a pc, slow on another pc

[luigi-c]

Hi all,
I have been using FreeFileSync since about two years for backup, it works fine
on my home pc (core2quad, Win 7 64-bit, external USB HDD).
On my work pc (PentiumD WinXP SP3, USB pendrive) it has worked fine until I
had to replace my old pendrive on last July, a 16 GB by Kodak, with a bigger
and faster one, a 32 GB A-Data S102. This pendrive has shown a 22 MB/s
sequential writing speed on my home pc, and 17 MB/s on my work pc;
nevertheless the backup on my work pc has become painfully slow.
As soon as I start the compare phase, the program seems to freeze, then goes
ahead, then freezes again. It takes several minutes to perform what required
less than a minute before the pendrive change. Sometimes I have to force it to
close, leaving sync.ffs files everywhere both on the HDD and the pendrive.
I tried to install several versions of FreeFileSync, but only with those
before 3.8 (3.3 for example) I got a not too slow behaviour.
Any idea?

[Zenju]

I can confirm this behavior! It seems the locking file somehow is the trigger
and I suspect invalidated buffers (managed by OS) due to writing the lock are
responsible. I'm investigating...

[luigi-c]

I am doing another test: I have replaced the A-Data pendrive with a 8 GB
Cruzer Micro, much slower then the A-Data.
Unbelievable, but true: FreeFileSync is working pretty fine, rather slow but
fine, without freezing, both in compare and sync phases.
USBDeview reports a 200 mA consumption of the Cruzer Micro, while the A-Data
consumption was 500 mA. I am wondering if it could depend on the USB ports.
Indeed on a much newer and more powerful pc, under Win 7, the A-Data pendrive
shows very good performance (more than 20 MB/s writing speed).

[Zenju]

The new USB drives usually have quite large buffers like 32 MB. Now if FFS
writes the lock file, even if its only a few bytes, it invalidates these
buffers and all the data that has been "read-ahead" is discarded and the
requested data needs to be read again. This is where the application stalls.
The stall itself is unrelated to FFS, you can notice this when you open the
drive in explorer while the USB's internal buffer is empty, like after a
system restart.
Older USB drives didn't have this problem due to smaller buffers; if such a
buffer is invalidated, only a few MB need to be read, which is faster than
reading 32MB with a new USB drive, so (almost) no stall.

[luigi-c]

After having completed the backup I tried to restart FFS but the new compare
phase has been very slow now, much slower than the first one.
The first backup I did on the empty Cruzer pendrive, the second on the same
pendrive but filled with the folders and files resulting from the first
backup.
Now FFS is synchronizing, but it's frozen at the 0.08% since several
minutes...

[luigi-c]

I downloaded the last version, 4.0, it works fine even on my slow work pc!
Well done!

[Zenju]

I haven't closed this topic yet, it is on my todo list of very few open
points. Luckily this perf issue is not dramatic, as is "only" affects binary
comparison against an usb memory stick with sufficient cache size (= three
criteria). This is quite a specific set of requirements. But still, perfection
demands, this should be solved :)

Btw, I'm glad you noticed the general speedup in v4.0. There have been a
couple of very sweet I/O optimizations!

[luigi-c]

You mentioned USB memory sticks, now I'm using an external USB hdd with FFS
4.0 and it works fine!
I haven't tried the 4.0 with a fast pendrive yet. Do you think the performance
would be seriously affected?

[Zenju]

The speedup in v4.0 is mostly about the synchronization step. As for the
comparison step, there is no difference.

[luigi-c]

With the previous 3.21 yet the comparison phase was terribly slow on my work
pc; two sync files appeared on the Windows desktop and the comparison process
stuck. On my home pc it has always been ok with all FFS versions.