RealTimeSync don't start during file reception

[lvloha]

HI,

I have a problem with RealTimeSync, I use it to make a mirror backup of a folder, everything works fine, but my problem is that the source folder receives files almost continuously (files with multiple images) and RealTimeSync does not start during copying, and does so only if the copy ends, there begins the synchronization, if not, he waits.

If I manually initiate synchronization with FreeFileSync even during a copy, synchronization starts, but not with RealTimeSync.

More details, if I copy several folder at the same time even if a folder is completed, the synchronization does not start, and expects that all folders are copied.

And it does not suit me, since I receive files continuously, synchronization will not begin until the end of the day.

Is there a setting to adjust, so that the RealTimeSync works in realtime? even during a copy ?

[Plerry]

See: viewtopic.php?t=1759

> Under "FreeFileSync - Help->RealtimeSync-#Overview->Limitations" is sais:
"While RealtimeSync is executing the command line, monitoring is inactive and
changes occurring during this time are not detected."

The help file probably sounds more scary than necessary. Some more explanation:

While RealtimeSync is executing the command line, monitoring for changed files is deliberately inactive.

The command line usually starts a synchronization task using FreeFileSync which naturally leads to additional file change notifications.
Therefore the RealtimeSync change detection has to be deactivated to not go into an endless loop.
On the other hand it is not likely that changes happen in first place since RealtimeSync runs the command line only after a user-specified idle time has passed which
makes sure the monitored folders are not in heavy use.
In any case, files changed during the execution of FreeFileSync will be synchronized the next time FreeFileSync runs.

[lvloha]

thank you Perry for your explanation, but it does not answer my question.

Return to my example :

If I "Copy" 4 folders containing 2,000 images each. (the RealTimeSync is pending but it not execut eny command line, the Idle time is 10 seconds), and i "Paste" to the main folder, copying starts (Windows Explorer) the RealTimeSync does not respond after 10 seconds, the copy continues... After 50 seconds a copy of a Folder is completed and passes the second folder (Windows Explorer), the RealTimeSync is still inactive, and remains so until that all the Folders are copied (2/3 minutes), there RealTimeSync starts synchronization.

And it does not suit me, because in my work, the copy can be non-stop.

[Plerry]

RealTimeSync is what it is, whether it suits you or not ...
If files are almost continuously being copied to the source folder(s),
RealTimeSync might not be the tool for you.

I also do not see what could or would need to be changed in RealTimeSync to fit your needs.

It seems you would benefit more from running FreeFileSync as a scheduled task,
e.g. every 15 mins, rather than using RealTimeSync to trigger running FreeFileSync.
But also then, files being copied to the source folder(s) while FreeFileSync is running
will only be synced to the target folder(s) the next time FreeFileSync will run.

[lvloha]

FreeFileSync synchronize files even while copying, that's why I Questione about RealTimeSync, but the idea of running FreeFileSync as a scheduled task can be interesting, I'll try.
Thank you very much.

[Plerry]

FreeFileSync synchronize files even while copying, ...

FreeFileSync will gladly run in parallel to other applications potentially writing to the source folder(s) of the FFS sync. However, new files written to the FFS source folder(s) once FFS has finished its comparison-phase (for that folder) will not be included in the ongoing FFS sync, but will only be detected as "to be synced" during the comparison-phase of the next FFS run.

By scheduling FFS to run at regular intervals, that next run is never more than the interval-time away, and does not, like when using RealTimeSync, depend on the folder(s) being idle (=not being modified or written to) for a given amount of time.

[Zenju]

If you don't have a time window of 10 seconds with no activity, then just reduce RealTimeSync's "minimal idle time" to whatever is barely workable. You could schedule FFS to run at fixed intervals, but it isn't generally a good idea to start a synchronization while some other tool is currently writing a file. With VSS deactivated you would get file copy errors, with VSS activated you would sync corrupt data. So usually you really want to benefit from RealTimeSync's key feature to "wait for a suitable time slot for sync", but you may need to adapt the default of 10 seconds to your scenario.

[ITmgr]

I realize this thread is a little dated, but I'm in the middle of discovering the impact of this issue to our organization's setup and had a couple of questions. Does RTS monitoring of folder pairs including subfolders? If so, does the inactivity period apply to the entire folder structure, or just the root folder? Is monitoring done with polling or by adding RTS to the folder change notification interrupt service?

My current setup monitors some very large (wide and deep) folder structures. I use RTS launched by Task Scheduler to run FFS batch files. I think I misunderstood the inactivity setting to apply at the file level, so my expectation was that any given file that was changed and then inactive for the timeout period would be individually synced. I see now that is not at all what happens. Even with a small inactivity window, some of the busier folder structures are not being synced throughout the day. But it goes deeper than that. I am seeing folder structures that are not synced in the evening, either, when I know they had file content that changed during the day. This is why the question about whether the monitoring is for sub folders. The behavior is as though only the root folder is being monitored, so if no changes happen at the root level, no sub folders are synchronized.

[ITmgr]

Follow up to my last post. After considerable testing, I observe there is a difference in RTS behavior when run from a login session and when run as a scheduled task. On the surface, it looks like when monitoring a folder on a local drive, a change in sub folders is only detected when RTS is running as a process launched from a console session. When launched by Task Scheduler, it seems to only be triggered when there is a change in the top level folder being monitored. I have been running the scheduled task as SYSTEM, so that may have an effect depending on how RTS knows when a change has occurred. I read somewhere that the folder change wait system does not return for subfolder changes when running as a system process. Is the effect I am seeing? Would creating a privileged user account to use with Task Scheduler instead of using SYSTEM address the issue, or is it because of the launch by Task Scheduler regardless of specified user?

[Zenju]

I've updated the manual to answer your first questions. Currently RTS has no nice UI to show which files have changed, but it is possible to get this information:
https://freefilesync.org/manual.php?topic=realtimesync

Whether changes are detected in top or sub folders should not depend from the user account that launched RTS. But the problem could be that RTS is not starting in first place (notification area icon turns grey) when the monitored folder is not accessible, and this can depend on the privileges that the user had that launched RTS.

[ITmgr]

Zenju, thank you for the update. After reading your manual update, and considerable testing, I can say that RTS does not trigger when a subfolder of the monitored folder changes if it was launched by Task Scheduler, whether as a SYSTEM or user account. This problem is very easy to reproduce, at least for Windows Server 2008. Maybe this is an OS shortcoming that has been fixed in later versions, but I have my doubts. Create an RTS setup. Launch it as a user and watch task manager - you will see the triggered command execute easily by adding or removing a file from the monitored root or sub folder. Exit RTS. Set up a scheduled job to launch RTS as either SYSTEM or the same user you just tested foreground launch with. Repeat the creation/deleting of a file in the monitored folder works, create/delete in a sub folder will not trigger RTS. This is a totally separate issue from the greyed out RTS icon waiting on folders to be available. Something about the context of the scheduled task is not receiving change notifications for subfolders. I've written hundreds of thousands of lines of Windows code, and seen this sort of odd behavior before for other system calls, particularly power management calls that work in a very similar fashion to change notifications. I suspect this is a Windows design issue, and nothing reasonable to be done about it.

I dealt with this issue by no longer using RTS, and just scheduling the same batch it would be running to run directly at timed intervals. It's not perfect, since it wastes resources this way checking to see if a sync is needed whether it might be or not, but at least I am confident I have my copies and the compare is so fast, I can live with the waste.

In summary, I would not call this an RTS bug yet. I more strongly a Microsoft issue. Whether they consider it a feature or a flaw is hard to say, but if all your code is doing is relying on system change notifications, and the behavior is different when launched by Task Scheduler than with launched from the UI, it's MS :)

As for the other issue, with the greyed out RTS never coming back, that may be another, different , MS issue. There are three RTS sessions running on the same machine, all monitoring different local folder structures. Two work, one does not, yet all three work perfectly when launched from the UI. Also very easy for me to repeat, but since I don't understand yet what is going on, I don't know how to tell you to repeat it. If you like, I could probably do a remote session and show you the behavior.