Syncing WSL symbolic links

[Ryamada]

I am on Windows and have started using WSL2 + Ubuntu.

I have tried to use FreeFileSync from Windows for sync'ing my WSL home between computers (home directory shared as WSL_home, and FFS pair = \\PC1\WSL_home , \\PC2\WSL_home).
For the most part it works superbly, carrying over the meta data that defines the Linux side permissioning.

Only problem seems to be symbolic links, which I'd like to sync/copy without following (using the Direct option, running FFS as administrator).

In a WSL session they are properly resolved, e.g.

nthome -> /mnt/c/Users/MyUserName
.vimrc -> my/vimrc


But attempting to sync them from Windows leads to errors such as:

Cannot resolve symbolic link "\\PC1\WSL_home\nthome".
Not a symbolic link or junction

Seen from Windows shell, the WSL symlinks are shown as junctions that cannot be resolved (target is shown as [...]):

2020-07-01  17:16    <JUNCTION>     .vimrc [...]
2019-11-16  11:52    <JUNCTION>     nthome [...] 


(looked up a bit on this, and apparently they are a WSL specific reparse point that's not a Win native junction or link ; also they appear to be possible to create without administrator or createsymboliclink privilege).

Any idea if it's possible to manage those without too much pain (= change the entire approach/toolset)?

[Zenju]

Any idea if it's possible to manage those without too much pain (= change the entire approach/toolset)?

FFS is definitively able to parse them. I don't have a test case myself, but I've written an implementation according to the documentation I've found. Does the following version work?
https://www.mediafire.com/file/5eaav9oid5fj0pw/FreeFileSync_10.26_%5BBeta%5D_Windows_Setup.exe

[Ryamada]

Thank you for the incredibly quick response!

And the beta version solves the parsing issue perfectly!
I could verify that everything works as intended if the destination is a local drive.

Unfortunately I still could not get it to work over network shares, even when running as admin, as I apparently do not have the privileges required to create the files on the remote drive. Error has become:

Cannot copy symbolic link
"\\PC1\WSL_home\nthome" to
"\\PC2\WSL_home\nthome".
ERROR_PRIVILEGE_NOT_HELD: A required privilege is not held by the client. [CreateSymbolicLink]

Not sure how to deal with that, but I reckon that it's not a FFS specific topic anymore.

Thanks again!

[Ryamada]

Actually one thing that's a bit unexpected is that the sync'ed links do not appear to be created as WSL reparse points.

In my local drive to local drive sync experiment, I also needed to be administrator for things to work (otherwise FFS protests with same above error), and once running as administrator, destination symlinks are created as native Windows symlinks.

In a CMD run as administrator I have:
Source =

2019-11-16  11:52    <JUNCTION>     nthome [...]


Destination, created by FFS =

2019-11-16  11:52    <SYMLINK>      nthome [/mnt/c/Users/MyUserName/]


Is it intended?

[Ryamada]

Apologies for replying to self again.

In case it's useful to anyone, I managed to make things work with my setup by:

- (to parse the WSL symlinks: ) using the FFS 10.26 beta

- (to allow FFS to create Windows symlinks on "local" PC1: ) running FFS as administrator on PC1

. (to allow FFS running on PC1 to create Windows symlinks on "remote" PC2: ) granting my user account on PC2 (same account name) the right to create symbolic links; running on PC2: Local Security Policy > User Rights Assignment > Create symbolic link -> add PC2\MyUserName
From reading around, this policy is supposed to be ineffective if user is part of the Administrators group (in which case they must run as administrator to be able to create Windows symlinks regardless of the policy). But apparently, it's effective (and required) if the machine's drives are accessed as a share (which is the case here).

I've done a few tests creating/deleting/modifying symlinks on both machines from WSL, always running FFS from PC1, and every time the symlinks have sync'ed properly between both machines.

Only caveat, as mentionned in previous message, is that symlinks created by FFS will be Windows symlinks, not WSL symlinks, but these work in WSL as well.
Eventually this results in having corresponding symlinks not being the same reparse point type on respective machines.

Ultimately, having FFS create WSL symlinks when source is itself a WSL symlink would be ideal (for consistency, and also because it does not appear to require elevation).
It would be great if it's feasible.

Still, fantastic job making this kind of sync'ing scenario work!

[Zenju]

Ultimately, having FFS create WSL symlinks when source is itself a WSL symlink would be ideal (for consistency, and also because it does not appear to require elevation).
It would be great if it's feasible.

Indeed, not needing admin rights is a real benefit. Also these WSL symlinks are probably of little use outside of WSL, so copying them as ordinary Symlinks doesn't add any value. Here's a version that copies them "as is". Does it work for you?
https://www.mediafire.com/file/at16vohbuod2ukx/FreeFileSync_10.26_%5BBeta%5D_Windows_Setup.exe

[Ryamada]

Thanks a lot for looking into this.

It seems to work perfectly if the pair are local drives (running FFS with no admin rights).

Unfortunately, not the case if the destination is a shared drive (regardless of running FFS with admin rights & having symlink creation allowed or not on remote machine).

The error is as follows:

Cannot copy symbolic link
"\\PC1\WSL_Home\nthome" to
"\\PC2\WSL_Home\nthome".

ERROR_ACCESS_DENIED: Access is denied. [DeviceIoControl(FSCTL_SET_REPARSE_POINT)]

Interestingly, FFS creates a 0 size file on the destination drive with the same name as the symlink before showing this error.

That file stays there even if ignoring/cancelling on the error message.

If attempting to retry upon receiving the error instead of ignoring/cancelling, the error message becomes:

Cannot copy symbolic link
"\\PC1\WSL_Home\nthome" to
"\\PC2\WSL_Home\nthome".

ERROR_FILE_EXISTS: The file exists. [CreateFile]

That file does not show up as a symlink in CMD or WSL shell:

Source / CMD (dir)

2019-11-16  11:52    <JUNCTION>     nthome [...]


Source / WSL shell (ls -l)

lrwxrwxrwx 1 MyUserName MyUserName  20 Nov 16  2019 nthome -> /mnt/c/Users/MyUserName/


Destination / CMD (dir)

2020-07-02  09:33                 0 nthome


Destination / WSL shell (ls -l)

-rwxr--r-- 1 MyUserName MyUserName    0 Jul  2 09:33 nthome


[Zenju]

ERROR_ACCESS_DENIED: Access is denied. [DeviceIoControl(FSCTL_SET_REPARSE_POINT)]
[...]
That file stays there even if ignoring/cancelling on the error message.

Maybe FSCTL_SET_REPARSE_POINT needs SE_RESTORE_NAME for this network location. Let's find out. Does symlink copying work when running the following version as admin? (The second problem was also fixed.)

https://www.mediafire.com/file/8k7kicvr9jj2huq/FreeFileSync_10.26_%5BBeta%5D_Windows_Setup.exe

[Ryamada]

Same error unfortunately.

I can confirm the 0 size file thing is fixed.

[Zenju]

(to allow FFS running on PC1 to create Windows symlinks on "remote" PC2: ) granting my user account on PC2 (same account name) the right to create symbolic links; running on PC2: Local Security Policy > User Rights Assignment > Create symbolic link -> add PC2\MyUserName

From reading around, this policy is supposed to be ineffective if user is part of the Administrators group (in which case they must run as administrator to be able to create Windows symlinks regardless of the policy). But apparently, it's effective (and required) if the machine's drives are accessed as a share (which is the case here).

A user as part of the admin group is not enough because UAC removes privileges when creating a non-elevated access token. So for the "Create symbolic link" right to be honored during remote access it's required to set up a new non-admin user account that has this right.

[Zenju]

It seems the reason why creating the WSL symlinks over network fails (while CreateSymbolicLink() OTOH succeeds) is because SMB implements special handling for IO_REPARSE_TAG_SYMLINK (= the standard NTFS symlink): https://malware.news/t/abusing-mount-points-over-the-smb-protocol/25820

[Zenju]

Are you able to create these WSL symlinks directly on the network share (without admin rights) from within a WSL session?

[Ryamada]

I tried and have not been able to.

What I did precisely: on local machine PC1, mount remote drive with:

$ sudo mount -t drvfs '\\PC2\WSL_home\'  /mnt/rhome -o metadata,uid=1001,gid=1001
$ cd /mnt/rhome


(permissions/ownership etc look correct, I can create files & directory, and follow existing WSL links properly - absolute remote WSL symlinks e.g. pointing to /mnt/c/someplace bring me back to local PC location if it exists but that's to be expected)

$ ln -s /mnt/c/Users/MyUserName nthome
ln: failed to create symbolic link 'nthome': Operation not permitted
$ touch dummyfile
$ ln -s dummyfile dummylink
ln: failed to create symbolic link 'dummylink': Operation not permitted


(so basically everything works fine except the symlink creation).

For sake of completeness, I have tried this with all combinations of the following:
- WSL run as administrator & as standar user
- granting or not my local & remote Windows account the Symbolic Link Creation priviledge (they are both part of Administrators group, I have not tried to change that)
- with & without ability to parse remote to local and remote to remote links via

fsutil behavior set SymlinkEvaluation R2L:1 R2R:1


[Zenju]

Thanks for the testing!

It seems the best behavior for FFS to adopt is to parse WSL symlinks and copy them as regular NTFS symlinks, even though it requires elevation (and additional set up if the target is a network share).

[Ryamada]

Indeed, it appears so, at least until WSL symlinks can be created on mounted network drives. Well that's a topic for the WSL/Windows teams - I'll try and find a place to ask about it.

Thanks so much for all the time looking into this & setting up the beta builds!
Very much appreciated!