Mirroring with symbolic links internally intact

[maphewyk]

I have about 1.5 TB of large satellite imagery files to periodically mirror to another location for redundancy. Within the Work volume many copies of the same files are referred to using symbolic and hard links.

Source:
A:\Reference\2019\July.tif
A:\Processing\Area_B\Link_to_2019July.tif -- Symlink to line above
A:\Processing\Area_B\Results.tif
A:\Deliverables\Area_B_Results.tif -- Hardlink to line above
A:\Processing\Area_C\Link_to_2019July.tif -- Symlink to 3 lines above
...etc.

Destination:
\\server\share\offsite_station_X\* -- all of the above

At present if I use FreeFileSync to mirror with symlink handling set to 'Follow' each of the sym- and hard-linked files are expanded and the source images copied, resulting in 3 and 4 times more storage consumed in the destination.

If I set link handling to 'Direct' the links are copied as links, which is great, however the target of those links is still the source volume. So

\\server\sha...\Area_B_Results.tif


resolves to

A:\Deliverables\Area_B_Results.tif


, which means the redundant copy is not truly a copy. If the source disappears the link breaks.

Is there any way to have FFS detect when a symlink and a target are both within the same source tree, and then create a link in the target that also stays within the same tree?

Environment is Windows 10 and Server 2019.

[xCSxXenon]

Well if you restored the backup, wouldn't those links work again?

[maphewyk]

That's a thought. It would require the mirror copy be restored to the same location which might not be possible on a different machine. It's still good to know they're not truly "lost" like I thought. It does mean the remote copy can't be explored in place which is a drawback.

[xCSxXenon]

Ah that is true... I'm sure there's a way FFS can determine the root location. But then would that mean the only way to restore is with FFS? How does Windows itself handle copying symlinks?

[maphewyk]

Yes in this scenario I think restoring would require FFS, but I'm not sure that "backup and restore" is the best way to think of this use case. It's more like "environment duplication". Environment B, the remote, is a copy of Env A, the local, but both of them are internally complete.

I'll do some experiments with native Windows copying and see what I learn about what it does.

[Zenju]

For the behavior you want, instead of an absolute symlink like

A:\Reference\2019\July.tif
A:\Processing\Area_B\Link_to_2019July.tif -- Symlink to line above


you could use a relative symlink to:

\Reference\2019\July.tif


The question is how much should FreeFileSync help in this regard.
It seems symlink content is basically "user data". I'm unsure if FreeFileSync should touch that.