Immediately after running a sync that copies tons of files, I run COMPARE again on the very same set. And again it tells me tons of files have changed, even though it just synced them.
It seems to be related to the old 1h problem: all the files that FFS just copied over, bear a different time stamp (1h ahead) than their namesakes on the other side.
However, when I look at these files in Windows or in any file manager, they carry the exact same time stamp as their counterparts. In other words, FFS seems to have a problem reading their time stamps correctly. I tried un- and re-plugging the drive, running a two-way sync instead of mirror - nothing helps. I can do it a hundred times over, it insists on those files being different.
The weird thing - it does not do this with all files, and even more confusing - not with all drives. With some drives it works correctly, with others it doesn't. All are FAT32 formatted, though. The computer side is NTFS Win7 x64
Won't recognize time stamp
- Posts: 85
- Joined: 28 Aug 2012
-
- Site Admin
- Posts: 7048
- Joined: 9 Dec 2007
Probably the following sequence of events happened in your case:
1. you synced files from or to a FAT32 drive before the last DST switch
2. after last DST switch you copied these files from FAT32 to NTFS via *Explorer*
3. when you now sync the files from FAT32 against the newly copied ones on NTFS you observe that FFS finds a 1 hour offset and even after sync with FFS, a second FFS comparison still shows this offset, but it's not visible in Explorer.
If so, this is a combined effect of FFS's DST handling and File System Tunneling.
1. you synced files from or to a FAT32 drive before the last DST switch
2. after last DST switch you copied these files from FAT32 to NTFS via *Explorer*
3. when you now sync the files from FAT32 against the newly copied ones on NTFS you observe that FFS finds a 1 hour offset and even after sync with FFS, a second FFS comparison still shows this offset, but it's not visible in Explorer.
If so, this is a combined effect of FFS's DST handling and File System Tunneling.
- Posts: 85
- Joined: 28 Aug 2012
Thank you for the explanation, Zenju
I don't keep track of when I copied what before or after the DST switch - but I trust your assessment, so this is probably what happened. Especially since it does not affect all files.
Are you saying I should NOT use Explorer to copy files?
But file managers also rely on Explorer under the hood - I cannot restrict files to being handled with FFS only.
I am not concerned with Explorer showing different time stamps than FFS.
The problem is that FFS flags so many files as different even though they are not, but FFS ignores that.
What am I to do?
Overwriting makes no difference. Do I have to delete the whole drives, then use FFS to fill them from scratch?
I don't keep track of when I copied what before or after the DST switch - but I trust your assessment, so this is probably what happened. Especially since it does not affect all files.
Are you saying I should NOT use Explorer to copy files?
But file managers also rely on Explorer under the hood - I cannot restrict files to being handled with FFS only.
I am not concerned with Explorer showing different time stamps than FFS.
The problem is that FFS flags so many files as different even though they are not, but FFS ignores that.
What am I to do?
Overwriting makes no difference. Do I have to delete the whole drives, then use FFS to fill them from scratch?
-
- Site Admin
- Posts: 7048
- Joined: 9 Dec 2007
> Are you saying I should NOT use Explorer to copy files?
Using Explorer is okay. Conceptually that is just like any other external change outside of FFS. The problem here is, due to the specific set of events described, you hit a corner case. Solving this case is a todo for a future release, but it's not easy (= not possible with the current design) and also not high priority.
> What am I to do?
Delete the problematic files on the FAT32 drive with Explorer, then copy the corresponding files back over from NTFS, either with FFS or Explorer.
Using Explorer is okay. Conceptually that is just like any other external change outside of FFS. The problem here is, due to the specific set of events described, you hit a corner case. Solving this case is a todo for a future release, but it's not easy (= not possible with the current design) and also not high priority.
> What am I to do?
Delete the problematic files on the FAT32 drive with Explorer, then copy the corresponding files back over from NTFS, either with FFS or Explorer.
- Posts: 85
- Joined: 28 Aug 2012
>Delete the problematic files on the FAT32 drive with Explorer, then copy the corresponding files back over from NTFS, either with FFS or Explorer.
Easier said than done. They're scattered all over the whole area of 600 GB. That means I first have to isolate them in FFS (since it's only here they show the different time stamps) - then find a way to tell Explorer to delete just these files.
And repeat the procedure twice a year when DST comes rolling around.
Hoping for a better solution in a future release - I really like FFS and was hoping to retire my other tools.
BTW, this seems to be the cause why FFS takes longer in comparison - it has nothing to do with copy speeds or analyzing (as I assumed in another thread), but only because it overwrites all these additional files.
Easier said than done. They're scattered all over the whole area of 600 GB. That means I first have to isolate them in FFS (since it's only here they show the different time stamps) - then find a way to tell Explorer to delete just these files.
And repeat the procedure twice a year when DST comes rolling around.
Hoping for a better solution in a future release - I really like FFS and was hoping to retire my other tools.
BTW, this seems to be the cause why FFS takes longer in comparison - it has nothing to do with copy speeds or analyzing (as I assumed in another thread), but only because it overwrites all these additional files.
-
- Site Admin
- Posts: 7048
- Joined: 9 Dec 2007
> then find a way to tell Explorer to delete just these files.
You can also delete the files manually from FFS.
> this seems to be the cause why FFS takes longer in comparison -
Unless you see superfluous "Encoding extended time information" during comparison, the comparison speed is unaffected by the DST handling.
You can also delete the files manually from FFS.
> this seems to be the cause why FFS takes longer in comparison -
Unless you see superfluous "Encoding extended time information" during comparison, the comparison speed is unaffected by the DST handling.
- Posts: 85
- Joined: 28 Aug 2012
>You can also delete the files manually from FFS.
The emphasis being "manually" (via right mouse DEL), correct? Not by flagging them for deletion via the middle column actions?
>comparison speed is unaffected by the DST handling.
I meant the extra time it takes to unecessarily copy all those files back and forth. The overall speed difference is significant (especially when large video files are involved, naturally).
It never occured to me to check up on this - usually I just let it run, and come back an hour later, sometimes suprised that it's still at work (when the above FAT32 situation applies). Which is why I kept using another tool for these heavy liftings. Until I discovered that those files are not really different at all, and that I could correct the action flags FFS had assigend to them. But that takes extra effort.
The emphasis being "manually" (via right mouse DEL), correct? Not by flagging them for deletion via the middle column actions?
>comparison speed is unaffected by the DST handling.
I meant the extra time it takes to unecessarily copy all those files back and forth. The overall speed difference is significant (especially when large video files are involved, naturally).
It never occured to me to check up on this - usually I just let it run, and come back an hour later, sometimes suprised that it's still at work (when the above FAT32 situation applies). Which is why I kept using another tool for these heavy liftings. Until I discovered that those files are not really different at all, and that I could correct the action flags FFS had assigend to them. But that takes extra effort.
-
- Site Admin
- Posts: 7048
- Joined: 9 Dec 2007
> The emphasis being "manually" (via right mouse DEL), correct?
Excactly, this is done to avoid the File System Tunneling effect which applies to deletion followed shortly by a file move, just like a regular sync would do.
Excactly, this is done to avoid the File System Tunneling effect which applies to deletion followed shortly by a file move, just like a regular sync would do.
- Posts: 85
- Joined: 28 Aug 2012
Thanks. This did the trick. Tried it on a rather small drive first. Right now it's busy syncing the big one (4TB)
Do you also have a solution for checking for 1h differences only? I am a bit scared to delete something that may indeed be newer on the other side. But I cannot possibly scan the whole list with my eyes to make sure I only have exact 1h diffs selected.
Do you also have a solution for checking for 1h differences only? I am a bit scared to delete something that may indeed be newer on the other side. But I cannot possibly scan the whole list with my eyes to make sure I only have exact 1h diffs selected.
-
- Site Admin
- Posts: 7048
- Joined: 9 Dec 2007
The best approach is to do a regular sync first, then compare again and list the files that are still different. Then delete them manually from the FAT side and finally setup a mirror sync or set sync direction manually to copy the files back from NTFS and start sync.
- Posts: 85
- Joined: 28 Aug 2012
Thank you.
Made a note of the procedure to do it right after the next DST switch - *sigh* - but to be honest - this is quite a show-stopper for large volumes.
Made a note of the procedure to do it right after the next DST switch - *sigh* - but to be honest - this is quite a show-stopper for large volumes.
-
- Posts: 2270
- Joined: 22 Aug 2012
Alternatively, you might consider to step away from FAT for the drives to be synced.
- Posts: 85
- Joined: 28 Aug 2012
>step away from FAT for the drives to be synced
Cannot do without FAT, just not feasible - the data does not stem from NTFS computers alone, but also other non-NTFS devices.
The problem compounds when you have a whole pool of machines, devices, drives, sticks ... which is exactly the reason I've come to rely on Sync-Software. A simple backup - that's easy - every file manager can handle that, nowadays.
But when you actually do exchange data between various devices - video cameras, viewer devices, TVs, handheld stuff - back and forth via huge backup drives - even syncing to that NTFS drive won't solve the problem. Some FAT files still keep their 1h difference when they end up on the NTFS.
Cannot do without FAT, just not feasible - the data does not stem from NTFS computers alone, but also other non-NTFS devices.
The problem compounds when you have a whole pool of machines, devices, drives, sticks ... which is exactly the reason I've come to rely on Sync-Software. A simple backup - that's easy - every file manager can handle that, nowadays.
But when you actually do exchange data between various devices - video cameras, viewer devices, TVs, handheld stuff - back and forth via huge backup drives - even syncing to that NTFS drive won't solve the problem. Some FAT files still keep their 1h difference when they end up on the NTFS.
-
- Site Admin
- Posts: 7048
- Joined: 9 Dec 2007
> Made a note of the procedure to do it right after the next DST switch
The problem should not happen again, unless the very exact steps 1, 2, 3 described above take place - which is very unlikely.
The problem should not happen again, unless the very exact steps 1, 2, 3 described above take place - which is very unlikely.
- Posts: 85
- Joined: 28 Aug 2012
Thanks Zenju,
Will wait for the next DST to come around and report back.
However, judging from your points 1, 2, 3 - it appears to me that the exact same thing is likely to happen again:
1 = true - I synced just now, with DST in effect
2 = true - in those months following the next DST switch - I will pull files off my external drive to take them with me on the road. If I am not allowed to use Win Explorer for copying them over, please let me know what file managers are compatible with FFS
3 = true - after editing some of these files or shooting new videos on the road, I will want to sync them back to my drive
Will wait for the next DST to come around and report back.
However, judging from your points 1, 2, 3 - it appears to me that the exact same thing is likely to happen again:
1 = true - I synced just now, with DST in effect
2 = true - in those months following the next DST switch - I will pull files off my external drive to take them with me on the road. If I am not allowed to use Win Explorer for copying them over, please let me know what file managers are compatible with FFS
3 = true - after editing some of these files or shooting new videos on the road, I will want to sync them back to my drive
