Mirroring with symbolic links internally intact

Discuss new features and functions
Posts: 7
Joined: 6 Mar 2017

maphewyk

I have about 1.5 TB of large satellite imagery files to periodically mirror to another location for redundancy. Within the Work volume many copies of the same files are referred to using symbolic and hard links.

Source:
A:\Reference\2019\July.tif
A:\Processing\Area_B\Link_to_2019July.tif -- Symlink to line above
A:\Processing\Area_B\Results.tif
A:\Deliverables\Area_B_Results.tif -- Hardlink to line above
A:\Processing\Area_C\Link_to_2019July.tif -- Symlink to 3 lines above
...etc.

Destination:
\\server\share\offsite_station_X\* -- all of the above

At present if I use FreeFileSync to mirror with symlink handling set to 'Follow' each of the sym- and hard-linked files are expanded and the source images copied, resulting in 3 and 4 times more storage consumed in the destination.

If I set link handling to 'Direct' the links are copied as links, which is great, however the target of those links is still the source volume. So
\\server\sha...\Area_B_Results.tif
resolves to
A:\Deliverables\Area_B_Results.tif
, which means the redundant copy is not truly a copy. If the source disappears the link breaks.

Is there any way to have FFS detect when a symlink and a target are both within the same source tree, and then create a link in the target that also stays within the same tree?

Environment is Windows 10 and Server 2019.
User avatar
Posts: 3551
Joined: 11 Jun 2019

xCSxXenon

Well if you restored the backup, wouldn't those links work again?
Posts: 7
Joined: 6 Mar 2017

maphewyk

That's a thought. It would require the mirror copy be restored to the same location which might not be possible on a different machine. It's still good to know they're not truly "lost" like I thought. It does mean the remote copy can't be explored in place which is a drawback.
User avatar
Posts: 3551
Joined: 11 Jun 2019

xCSxXenon

Ah that is true... I'm sure there's a way FFS can determine the root location. But then would that mean the only way to restore is with FFS? How does Windows itself handle copying symlinks?
Posts: 7
Joined: 6 Mar 2017

maphewyk

Yes in this scenario I think restoring would require FFS, but I'm not sure that "backup and restore" is the best way to think of this use case. It's more like "environment duplication". Environment B, the remote, is a copy of Env A, the local, but both of them are internally complete.

I'll do some experiments with native Windows copying and see what I learn about what it does.
User avatar
Site Admin
Posts: 7040
Joined: 9 Dec 2007

Zenju

For the behavior you want, instead of an absolute symlink like
A:\Reference\2019\July.tif
A:\Processing\Area_B\Link_to_2019July.tif -- Symlink to line above
you could use a relative symlink to:
\Reference\2019\July.tif
The question is how much should FreeFileSync help in this regard.
It seems symlink content is basically "user data". I'm unsure if FreeFileSync should touch that.