Ubuntu 14.04 sync to NTFS: Cannot write modification time of xxxx (Error Code 22: Invalid argument (utimes))

[toxaris01]

I am unable to sync between my Ubuntu server and my mediaplayer. I get the errors like the one in the subject header. It seems that it fails when the time of the file must be adjusted (of the .ffs_tmp files to be exact). From what I have gathered, it might be a problem of the implementation of NTFS/CIFS on Linux. However, if I try to change the modification date of a file with the touch command (touch -d), it works without a problem. As a workaround, would it be possible to use touch instead of utimes?

[toxaris01]

A quick additional note, touch uses utimensat instead of utimes on Ubuntu. So, perhaps if utimensat instead of utimes was used, it would also work.

[Zenju]

I've seen utimensat fail silently for NTFS volumes on Ubuntu, while utimes worked fine. It seems the most widely supported API is utime, but it doesn't support symbolic links, which is why FFS uses utimes instead. Also utime is superseeded by utimes according to the Posix standard. So the best(?) API to use on Linux seems to be utimes.

[toxaris01]

It may be, although it does not seem to work on Ubunty with CIFS/NTFS. Perhaps a fallback option? I like the program on Windows, but this issue makes it unusable to use on my server it seems.

[Zenju]

Dos this happen for all the files or only specific ones? It seems somewhat unlikely that utimes would not be supported, maybe the problem is really an "Invalid argument".

[Zenju]

I've created a test version with some enhanced error message. Can you post the error message you see with this one?
[404, Invalid URL: https://freefilesync.org/FreeFileSync_6.15_beta_Ubuntu_14.10_64-bit.tar.gz]

[toxaris01]

Ok, I have tested the version you posted. I had to install some additional libraries first, but no real issue. Some icons do not have pictures btw.

I get the following error:
Cannot write modification time of "/media/m8/hdd/tv/The Block All Stars/Season 10/The Block - S10E22 - Bedrooms Revealed.mp4.ffs_tmp".

Error Code 22: Invalid argument (utimes)
utimes(pathname = "/media/m8/hdd/tv/The Block All Stars/Season 10/The Block - S10E22 - Bedrooms Revealed.mp4.ffs_tmp", data = {{ 1.4252882e+09 seconds }, { 1.4251831e+09 seconds }}) failed, Invalid argument (22, EINVAL)

I checked other files (big and small), but the error is the same.

[Zenju]

Okay, unfortunately the "explain_utimes" function fails to deliver any useful insights why the call fails.

I guess there is one last thing we can try: Let's use "utime" and see if this works:
[404, Invalid URL: https://freefilesync.org/FreeFileSync_6.15_beta2_Ubuntu_14.10_64-bit.tar.gz]

[toxaris01]

I get the following error:

Cannot write modification time of "/media/m8/hdd/tv/2 Broke Girls/Season 01/2 Broke Girls - S01E01 - Pilot.avi.ffs_tmp".

Error Code 22: Invalid argument (utime)

[Zenju]

I made a few tests with utimensat, but couldn't reproduce the silent failure bugs when syncing against NTFS as observed back in 2013. Maybe these bugs have been fixed meanwhile.
Other people filed bug reports, too, but it's impossible to tell if the issues still exist without testing the change in the next release.
"touch" using "utimensat" is a strong indication that the system trusts this API, maybe we should give it another chance. Here's the new beta implementing utimensat. Let me know if you find any issues:

[404, Invalid URL: https://freefilesync.org/FreeFileSync_6.15_beta3_Ubuntu_14.10_64-bit.tar.gz]

[toxaris01]

Alas...

Cannot write modification time of "/media/m8/hdd/tv/2 Broke Girls/Season 01/2 Broke Girls - S01E01 - Pilot.avi.ffs_tmp".

Error Code 22: Invalid argument (utimensat)

[toxaris01]

Could it be that the call is incorrect?

[Zenju]

Now this is surprising! You mentioned that the "touch" command worked correctly on a file on volume "/media/m8/"?

Can you log two traces:

1. setup a minimal sync job, which copies a single file only, then run it while creating a log file

strace ./FreeFileSync &> ffs_log.txt

2. a second trace when doing a "touch" on the very same file at the target volume:

strace touch yourfilename.dat &> touch_log.txt

[toxaris01]

Sure:

https://dl.dropboxusercontent.com/u/10558681/ffs_log.txt
https://dl.dropboxusercontent.com/u/10558681/touch_log.txt
https://dl.dropboxusercontent.com/u/10558681/touch1_log.txt

In the last one I use command 'touch -d "two hours ago" <filename>

[Zenju]

Okay, I see three major differences between touch and FFS. Assuming the EINVAL is caused by only one of them, let's test each one in isolation:

1. don't use UTIME_OMIT, but set current time for last access:
[404, Invalid URL: https://freefilesync.org/FreeFileSync_6.15_beta_10_Ubuntu_14.10_64-bit.tar.gz]

2. open file handle manually and use futimensat
[404, Invalid URL: https://freefilesync.org/FreeFileSync_6.15_beta_11_Ubuntu_14.10_64-bit.tar.gz]

3. FFS uses the absolute path, while the touch example uses a relative one. A bit far-fetched, but maybe there is a problem when using touch with the absolute name?

[toxaris01]

Version 2 seems to work. I only has the following message twice during the whole operation (in the terminal):

ALSA lib pcm.c:7843:(snd_pcm_recover) underrun occurred

[Zenju]

Interesting, the only difference between beta3 and beta11 is that a handle is opened manually in the latter case. So whatever the problem is, it's certainly not an invalid argument, but merely bad error reporting. What the developer of the CIFS implementation meant to say was probably ENOSYS instead.
Anyway, I don't like cargo-culting around the peculiar implementation details of CIFS, so let's see if there is a more conformant solution:

1. Are you able to "cp --preserve=timestamps" the test file to the media player via command line?
2. If yes, can you post the corresponding strace to see how modification time is copied?

[toxaris01]

Copy with the preserve timestamp works. Here is the strace:
https://dl.dropboxusercontent.com/u/10558681/cp_log.txt

[Zenju]

Okay, FFS will follow the "touch" and "cp" implementation in order to work around these issues with incomplete/insufficiently tested file system implementations. Here is the "final" version implementing these changes:

[404, Invalid URL: https://freefilesync.org/FreeFileSync_6.15_beta_Ubuntu_14.10_64-bit.tar.gz]

Thanks for your logs and feedback!

[toxaris01]

No problem! The 'final' works great!

[vitex2011]

Hi, hope it is okay to add to this.I have been struggling for a couple of months with both Freefilesync and file managers when sychronising and copying files (with all ending up with today's time and date or refusing to write due to utimes). At least with me, the problem stemmed from doing so with an NTFS partition under Ubuntu 14.04 (64 bit), and how it was mounted with respect to file attributes/permissions. The solution was to edit the fstab file in root manually. Mine had the entry 'UUID=01D034A4EFB3E710 /mnt/915\040Gi\040Files\0402 ntfs defaults,umask=007,gid=46 0 0 ' and the problem was solved when I took out the ',umask=007,gid=46' part of it. See http://ubuntuforums.org/showthread.php?t=1447350&highlight=copy+preserve+with+GUI which indicated the solution. Mine is a dual boot machine, so this must affect many people who are not so familiar with Ubuntu/Linux, perhaps you could let the relevant Ubuntu people know so that this issue is avoided by default when mounting NTFS partitions/drives. I am not sure if it affects other non EXT filesystems.

[toxaris01]

I knew about that, but this is not a physical drive in the server, but on another machine. The solution described does not work in this case. The version I tested the last time worked fine!