Fails to access file info with Umlaut U in OSX Sierra

[jimbtek]

When doing a folder comparison looking at a local Mac folder (running FFS on the Mac) and comparing to a network Linux NAS folder, it fails to access file info on the local Mac Files that contain the Umlaut U in the file name. I am in the US but had a client with a German name.

FFS 8.9 (downloaded it today)
Mac OSX Sierra 10.12.3

[Zenju]

What error message is shown?

[jimbtek]

Cannot read file attributes of "(filename with ü in it)".

Error Code 2: No such file or directory [lstat]

It does this consistently on all ~20 files with the umlaut, even in a few different directories, while not erroring on any other files in the same directories. This is out of ~200,000 files that i am comparing and there are no other errors.

[Zenju]

This is a unicode decomposition issue. Are you able to view the file in Finder on your Mac?

[jimbtek]

Yes I can view them in Finder and open the files. they are PDFs, PNGs, XLSX, etc.

[jimbtek]

i can see them in Terminal too

[Zenju]

Interesting. I'm able to reproduce the same error when accessing a Linux-hosted Samba share that contains decomposed file names from macOS. However these files are not shown by Finder. Since decomposed Unicode is the standard on macOS, this seems to be a bug in the Samba integration. Precomposed Unicode OTOH works fine. Considering that FFS normalizes all file names to decomposed, the Samba bug seems to be that all file names are normalized to precomposed in the Samba access layer. In this case there is nothing that FFS could do to workaround the issue.

In your test case, are these non ASCII chars decomposed or precomposed Unicode?

[jimbtek]

Hey Zenju,

Tell me how to figure that out :)

Also I have access to linux, windows, and Mac if you want me to try the same files but moved to a different OS. I can't recall if they were originally created on a Mac, but they have definitely moved from Mac to Win to Mac and are on Dropbox now as well.

[Zenju]

Easy, just go to the directory containing the file on Linux, enter

ls -l | od -xc


and... better just paste the result here ;)

[jimbtek]

OK, this is 3 files 2 PDFs, 1 JPG
-----------------

0000000 6f74 6174 206c 3031 3630 0a34 722d 7877
t o t a l 1 0 0 6 4 \n - r w x
0000020 2d2d 2d2d 2d2d 2040 2031 696a 206d 7720
- - - - - - @ 1 j i m w
0000040 6568 6c65 2020 2020 3620 3433 2032 6546
h e e l 6 3 4 2 F e
0000060 2062 3432 2020 3032 3531 5720 cc75 7288
b 2 4 2 0 1 5 W u ̈ ** r
0000100 6874 532d 3146 492d 6f63 2e6e 706a 0a67
t h - S F 1 - I c o n . j p g \n
0000120 722d 7877 2d2d 2d2d 2d2d 2040 2031 696a
- r w x - - - - - - @ 1 j i
0000140 206d 7720 6568 6c65 2020 3834 3932 3037
m w h e e l 4 8 2 9 7 0
0000160 2039 7041 2072 3032 2020 3032 3531 5720
9 A p r 2 0 2 0 1 5 W
0000200 cc75 7288 6874 532d 6c61 7365 542d 6172
u ̈ ** r t h - S a l e s - T r a
0000220 6e69 6e69 2d67 3230 702e 6664 2d0a 7772
i n i n g - 0 2 . p d f \n - r w
0000240 2d78 2d2d 2d2d 402d 3120 6a20 6d69 2020
x - - - - - - @ 1 j i m
0000260 6877 6565 206c 2020 3033 3138 3730 4d20
w h e e l 3 0 8 1 0 7 M
0000300 7261 3220 2033 3220 3130 2035 7557 88cc
a r 2 3 2 0 1 5 W u ̈ **
0000320 7472 5f68 6843 6e61 6567 4f5f 6472 7265
r t h _ C h a n g e _ O r d e r
0000340 305f 2e32 6470 0a66
_ 0 2 . p d f \n
0000350


---------------

[Zenju]

The issue is indeed with decomposed Unicode:

W  u  Ì  ˆ  r  t  h
57 75 cc 88 72 74 68 


Precomposed would have been:

W  à ¼  r  t  h
57 c3 bc 72 74 68