The parameter is incorrect. (DeviceIoControl, FSCTL_SET_SPARSE)

[tomtheone]

Hi all,

I try to copy files from a source (NAS device, CIFS shares: HP PolyServ cluster) to anoter NAS, also CIFS shares (unknown storage).

If i try to mirror the files, i get this error in FreeFileSync:

Cannot write file attributes of "\\ictcmsswp14\share1\A001\20141020\041G12A0012014102008271700.ffs_tmp".
Error Code 87: The parameter is incorrect. (DeviceIoControl, FSCTL_SET_SPARSE)

Disabling fail-safe copy at the options does not make a difference.
I was not able to find a option to disable the copy of file attributes, there is also no information about this topic at 'help'. Is this possible?
I'm using version 6.11.
I was using robocopy before, that was working obviously with those parameters: /COPY:DAT /E /MIR
I installed FreeFileSync on Win2k3 - 64bit.

If someone has a hint, please let me know.

Br,
Tom

[Zenju]

There's something fishy going on: FFS automatically copies sparse files, but only if the target has FILE_SUPPORTS_SPARSE_FILES set. So in your case it seems this flag is set, but it does not support sparse files? Do you know some more details about your target file system?

[tomtheone]

Hi Zenju,

Thank you for your feedback.

I think it's a HP 3PAR SAN with a related Windows Storage Server 2012 NAS Head. I try to organize the details about the target storage and also about the target FileSystem. Keep you updated.



Br,
Tom

[tomtheone]

Hi Zenju,

I got a confirmation that my destination is a Windows 2012 Server with NTFS filesystem.

Question: If it's not supported, can i disable the use of SPARSE_FILES?

Br,
Tom

[Zenju]

> I installed FreeFileSync on Win2k3 - 64bit.

I made a few tests on Win2003 - 64bit, but could not get FreeFileSync 6.11 to sync at all! The symptoms were extremely weird: a hang due to GDI leaks!
The underlying root cause are new assembly instructions that are not supported on this system. However the 32-bit binary, located in "bin\FreeFileSync_Win32.exe" works fine.

Maybe your problem is related? Does the 32-bit executable work for you?

[tomtheone]

Hi Zenju,

I made a misstake, i work on 32bit.

Host Name: SRPDX2
OS Name: Microsoft(R) Windows(R) Server 2003, Standard Edition

OS Version: 5.2.3790 Service Pack 2 Build 3790
OS Manufacturer: Microsoft Corporation
OS Configuration: Standalone Server
OS Build Type: Multiprocessor Free
Registered Owner: UHBS
Registered Organization: UHBS
Product ID: 69712-OEM-4411902-02112
Original Install Date: 16.12.2011, 09:57:11
System Up Time: 3 Days, 20 Hours, 27 Minutes, 52 Seconds
System Manufacturer: HP
System Model: ProLiant DL360 G7
System Type: X86-based PC
Processor(s): 12 Processor(s) Installed.
[01]: x86 Family 6 Model 44 Stepping 2 GenuineIntel ~
2666 Mhz
[02]: x86 Family 6 Model 44 Stepping 2 GenuineIntel ~
2666 Mhz
[03]: x86 Family 6 Model 44 Stepping 2 GenuineIntel ~
2666 Mhz
[04]: x86 Family 6 Model 44 Stepping 2 GenuineIntel ~
2666 Mhz
[05]: x86 Family 6 Model 44 Stepping 2 GenuineIntel ~
2666 Mhz
[06]: x86 Family 6 Model 44 Stepping 2 GenuineIntel ~
2666 Mhz
[07]: x86 Family 6 Model 44 Stepping 2 GenuineIntel ~
2666 Mhz
[08]: x86 Family 6 Model 44 Stepping 2 GenuineIntel ~
2666 Mhz
[09]: x86 Family 6 Model 44 Stepping 2 GenuineIntel ~
2666 Mhz
[10]: x86 Family 6 Model 44 Stepping 2 GenuineIntel ~
2666 Mhz
[11]: x86 Family 6 Model 44 Stepping 2 GenuineIntel ~
2666 Mhz
[12]: x86 Family 6 Model 44 Stepping 2 GenuineIntel ~
2666 Mhz
BIOS Version: HP - 2
Windows Directory: C:\WINDOWS
System Directory: C:\WINDOWS\system32
Boot Device: \Device\HarddiskVolume1
System Locale: de-ch;German (Switzerland)
Input Locale: de-ch;German (Switzerland)
Time Zone: (GMT+01:00) Amsterdam, Berlin, Bern, Rome, Stockholm,
Vienna
Total Physical Memory: 4'096 MB
Available Physical Memory: 2'418 MB
Page File: Max Size: 8'126 MB
Page File: Available: 6'667 MB
Page File: In Use: 1'459 MB
Page File Location(s): c:\pagefile.sys
Domain: CENTRICITY
Logon Server: \\SRPDX2
Hotfix(s): 144 Hotfix(s) Installed.

I think this is more related to the Windows 2012 destination, because when i switch the destination to something local or to another NAS CIFS share, then it's working.

If you are interested in troubleshooting, what do you exactly need?

Br,
Tom

Btw: Did you see this article: http://msdn.microsoft.com/en-us/library/windows/desktop/aa364596.aspx ?

[Zenju]

> If you are interested in troubleshooting, what do you exactly need?

Steps to reproduce the issue on my end. I've made a few tests sharing a folder on a Windows 2012 server using ordinary Windows network share and accessing it from Server 2003 x64. But I could not find any issues, i.e. copying sparse files from and to this network share worked correctly.

>Btw: Did you see this article: http://msdn.microsoft.com/en-us/library/windows/desktop/aa364596.aspx

Yes, what about it?

[wolfdogg]

Error Cannot write file attributes of "\\MERLIN\san\redtail\data\dpub\Downloads\photoshop lightroom plugins\Lightroom Presets 03\300 v2.lrtemplate.ffs_tmp".
Error Code 1: Incorrect function. (DeviceIoControl, FSCTL_SET_SPARSE)
...
3:08:15 AM Error Cannot write file attributes of "\\MERLIN\san\redtail\data\dpub\Downloads\photoshop lightroom plugins\Lightroom Presets 03\Colorful HDR Effect 1 - LightroomLibrary.lrtemplate.ffs_tmp".
Error Code 1: Incorrect function. (DeviceIoControl, FSCTL_SET_SPARSE)

i have been plagued by this issue for a few months now. I have switched out samba shares from one arch server, to a new rhel server, they are hosted on zfs raidz2. when pushed to from win7x4ult i get this error. Im seeing it again today after i finally got things back online, and backup 'almost' complete.

For the desperate: I found that if i delete the files on the backup, i can then atleast complete the backup.

Im suffering this, and the CreateSymbolicLinkW error. I am running freefilesynnc as administrator from a non admin account. When started, and going to file open, its admin, accesses admin documents path file saves, runs as admin, still this error.

Im going to try to run it from a multi-concurrent admin rdc login, like to do also sometimes, to see if that resolves the CreateSymbolicLinkW error atleast.

The sparse error i thought was zfs drivers (see bug report, and linked post here https://aur.archlinux.org/packages/zfs-linux-git/) , however using RHEL over arch and understanding more that this is a windows error message, from an unsupported file system feature, im pretty sure its not zfs related.

Is there a setting i can toggle on ffs, can this be related to versioning, file permissions, or any leads on what i can do get my backup to complete would be great.

Any suggestions on a fix there im all ears, im really worried about my 3TB data, for months now.

[Zenju]

Your assumption is correct, the target file system has flag "FILE_SUPPORTS_SPARSE_FILES" set, which is obviously a lie => file system implementation bug. The source file has "FILE_ATTRIBUTE_SPARSE_FILE" set, otherwise FFS wouldn't have tried to copy the file as sparse. Maybe this also is reported incorrectly by the source file system.

CreateSymbolicLinkW expectedly fails since the target file system is not on Windows.

[wolfdogg]

HI Zenju, thanks for getting back to me,
Im going to try a multi pronged approach on this. I ired off a bug reporthttps://github.com/zfsonlinux/zfs then i looked into this further.

So what im seeing on the docs here i had an idea. Based on this line on this page
"You can determine whether there are any sparse regions in a file by using the FSCTL_QUERY_ALLOCATED_RANGES , or control code."
https://msdn.microsoft.com/en-us/library/windows/desktop/aa364596(v=vs.85).aspx

or FILE_SUPPORTS_SPARSE_FILES
https://blogs.msdn.microsoft.com/openspecification/2010/11/05/notes-on-sparse-files-and-file-sharing/ or is that the one thats coming back TRUE? How so if i dont see anything such on zfs properties?

I was thinking what if you checked for that first, and and if found, just simply delete the file, so it can do a clean full write (put a conditional in there if on SMB, linux, zfs, whatever, etc..), since i have not seen this condition anywhere on the internet yet, excepct from coming out of your bleeding edge software (im long this, real engineering going on here).

Edit:
Ok i better explain, i was basing this on the assumption that somehow a sparse file is there, but how could it be in the first place if you say zfs doesnt support it. Maybe it just doesnt support the operation of, but has some on its drive, from a ffs push, and now cant get them managed since they are gridlocked... just a thought.

Mind you, i used this sytem for almost a year prior with no issues until that update on or near Oct27th-2016 win7-x64 version. The thing is, i cant pin it down if it was a ffs update, or the changed zfs array, from a 3 drive linear zfs jbod pool, to a 6 disk raidz2, so yeah i did change the array type, which means it did inherit a whole new set of properties 2 years after the establishment of the prior jbod array. So there may have been a different in a property somewhere. Man i wish i can dig up my old props from that array, guess i should start backing them up. They are linked in the post, so there we go.

[Zenju]

The bug probably isn't at the file system level (zfs), but one layer above within the software (Samba?) that wraps the Linux file system for consumption by a Windows OS. This software layer hasn't implemented FSCTL_SET_SPARSE, but still reports FILE_SUPPORTS_SPARSE_FILES, which is a contradiction.

[wolfdogg]

It looks like ZFS does support sparse files. They adopted this SEEK_HOLE and DATA_SEEK way back in 2006 or something https://blogs.oracle.com/bonwick/en_US/entry/seek_hole_and_seek_data#comment-1156250851000 , which i think has something to do with lseek, and now im trying to find out if just maybe when i created the zfs dataset i didnt need to actually make it sparse itself, so that it handles sparse files... e.g. zfs create -s -V ...

I found that there is a way to tell samba not to use sparse files, and an aid in helping not to follow symlinks, or vice ersa.
(follow symlinks, wide links, share:fake_fcscaps = 0, and the seeming most important one, which also isnt helping, strict allocate = yes)

this hasnt solved my issue at all though, error persists on any files that think they are sparse, or are sparse, and think they are not, or what? Im not really sure which is it, so im not sure how to combat it. It feels as if ntfs has sparse files, samba configs dont make any differnce, and that zfs is supposed to support it, just that i dont have it set up right, and ffs and zfs are having a miscommunication for some unbeknownst reason, but i could be wrong.

[wolfdogg]

two new ref urls here
http://unix.stackexchange.com/questions/345097/do-i-format-the-zfs-file-system-or-just-mount-and-share
https://lists.samba.org/archive/samba/2017-February/206580.html

Zenju, would mind taking another look at this? Im wondering if freefilesync is following the rules when it comes to talking to samba, or if samba is able to convey the rules given to it by ffs, to the native linux filesystems (zfs-on-linux in this case)

Let me know if i have misstated anything in those posts that are linked, i can do a cleanup run, since this is technical, i want to make sure im putting respect where respect is deserved, and its only because of your wonderful app that im able to even be spending my months on getting this to work, since nothing else has (think synctoy, im a lightweight app guy, thats the only other lightweight app i know of thats so configurable in task scheduler, etc... way outdated.. inferior to FFS).

[Zenju]

Zenju, would mind taking another look at this? Im wondering if freefilesync is following the rules when it comes to talking to samba, or if samba is able to convey the rules given to it by ffs, to the native linux filesystems (zfs-on-linux in this case)

There are no Samba-specific rules, but Samba must properly implement the Windows file system API which is used by FFS. Returning success for FILE_SUPPORTS_SPARSE_FILES, but fail for DeviceIoControl, FSCTL_SET_SPARSE is simply a Samba bug. The fix is trivial: Samba's implementation for GetVolumeInformation should not return FILE_SUPPORTS_SPARSE_FILES or just support FSCTL_SET_SPARSE. https://msdn.microsoft.com/en-us/library/windows/desktop/aa365566

[cestbienmoi]

After 2+ years of this problem, finally i find the solution, i add :
store dos attributes = yes
to the smb.conf (this can impact performance) (this is no by default for standalone samba server, see https://wiki.samba.org/index.php/Setting_up_a_Share_Using_Windows_ACLs for more info).

For complete information, my previous solution was to replace the sparse file to a plain file on the source. And disable sparse file for qbittorrent that is on by default !

[Zenju]

/etc/samba/smb.conf:

[global]
store dos attributes = yes


...does not work in my tests. Yes, it makes the "DeviceIoControl, FSCTL_SET_SPARSE" call succeed, but now BackupWrite will fail with a mysterious ERROR_FILE_NOT_FOUND.

For the next FFS version I've implemented checks to detect buggy SMB implementations (interestingly there is no such problem on macOS' SMB) and skip copying files as sparse.