It's there! (FTP) But with some bugs

Posts: 5 · downloadgemist 25 Apr 2017, 19:04

Hello Zenju,

I see you used my test FTP to good use... ;)
Great to see you implemented FTP support! I made a donation for the great work.

But during the first tryout syncing one server to another, I experienced two bugs. The first is encoding related error. A folder with in the name "zürich" could not be retrieved. It had something to do with the u umlaut, so I expect other special characters will also result in errors.

: Encoding error; encoding error.jpg (25.31 KiB) Viewed 3382 times

The second error was more difficult to overcome. During syncing, the process got itself stuck on a large file. The CPU rose to 100% and the transfer speed dropped to a dozen KB's per second.

: heavy load and low throughput; high cpu usage low transfer speed bug.jpg (131.5 KiB) Viewed 3382 times

The environment in which this happened was:
Windows server 2008 build 6002
The server from which was synced was the same operating system and the same build.
Syncing was done through FTP (not secure), without encryption.

Hopefully this information is helpfull.
Greetz!

Posts: 7052 · Zenju 25 Apr 2017, 20:50

But during the first tryout syncing one server to another, I experienced two bugs. The first is encoding related error. A folder with in the name "zürich" could not be retrieved. It had something to do with the u umlaut, so I expect other special characters will also result in errors. downloadgemist, 25 Apr 2017, 19:04

I presume the problem is a non-compliant server that is not using the UTF8 encoding but rather some local ANSII encoding. Other FTP software simply chooses to not support such legacy servers. But maybe there is some way to detect the encoding? If yes, I could integrate it into FFS.

The second error was more difficult to overcome. During syncing, the process got itself stuck on a large file. The CPU rose to 100% and the transfer speed dropped to a dozen KB's per second. downloadgemist, 25 Apr 2017, 19:04

If you send me one or better a few crash dump files I might be able to determine what FFS was doing while it was hogging CPU: https://freefilesync.org/faq.php#crash

Posts: 5 · downloadgemist 25 Apr 2017, 21:02

If you send me one or better a few crash dump files I might be able to determine what FFS was doing while it was hogging CPU: https://freefilesync.org/faq.php#crash Zenju, 25 Apr 2017, 20:50

I will create some dumps when the process slows down. However, it never completely hangs or crashes. It just goes from a solid transferspeed to a nihil transfer speed and high CPU usage.

I presume the problem is a non-compliant server that is not using the UTF8 encoding but rather some local ANSII encoding. Other FTP software simply chooses to not support such legacy servers. But maybe there is some way to detect the encoding? If yes, I could integrate it into FFS. Zenju, 25 Apr 2017, 20:50

I also tested the folder transfer with filezilla, to see if it also choked on the "ü", but it did not (while running on the same windows server 2008). Maybe Filezilla reads more encodings.
I do however not know if windows server 2008 does not support UTF8 encoding or not. Is there some way to check which encoding is set for an FTP transfer?

(character encodings are always a pain in the ass if you ask me :) )

Posts: 7052 · Zenju 25 Apr 2017, 21:11

I will create some dumps when the process slows down. However, it never completely hangs or crashes. It just goes from a solid transferspeed to a nihil transfer speed and high CPU usage. downloadgemist, 25 Apr 2017, 21:02

Still strange enough so that it's worth to analyze. There should not be any "busy waiting", but it's possible it's originating from the FTP access layer (but it still should be fixed).

I also tested the folder transfer with filezilla, to see if it also choked on the "ü", but it did not (while running on the same windows server 2008). Maybe Filezilla reads more encodings.
I do however not know if windows server 2008 does not support UTF8 encoding or not. Is there some way to check which encoding is set for an FTP transfer?
(character encodings are always a pain in the ass if you ask me :) ) downloadgemist, 25 Apr 2017, 21:02

With FTP I'm mentally prepared for the worst, so bring it on! ;) Okay, FFS should never perform worse than FileZilla, so I'd like to have a look. Can you again share a few test credentials for this server?

Posts: 7052 · Zenju 25 Apr 2017, 21:12

Just realizing: the problem with "ü" could also be due to this already fixed bug:
viewtopic.php?t=4292

Posts: 5 · downloadgemist 25 Apr 2017, 21:32

I read there is a beta to test this with? (the ü bug thingy). I will also provide you with the credentials. However, as you maybe figured, we're in the middle of a server transition. So the credentials will be tomorrow :)

Posts: 5 · downloadgemist 28 Apr 2017, 15:48

Just finished setting up the testFTP (with temporary subfolders and files, all kind of stuff to work or not work). The new Beta did not do anything differently, still had problems with all the special characters. I also tested the beta on Windows 2008 as well on Windows 10. There was no difference.
Windows explorer itself and FileZilla both still read all the folders and files.

I will send you the test credentials.

I still have to make the dumpfiles for you for the other problem. ;)

Posts: 7052 · Zenju 29 Apr 2017, 13:09

The character encoding issue has been fixed! The server is RFC-2640-non-compliant (UTF8 off by default), but there is a workaround that FFS now applies: http://www.mediafire.com/file/f8s51i8r9e95l8c/FreeFileSync_9.1_beta_Windows_Setup.exe

Posts: 5 · downloadgemist 2 May 2017, 07:50

That's quick :) I'll test it out soon and will let you know.

Posts: 6 · monin 4 May 2017, 19:23

Thanks for the beta, but unfortunately the same problem with the accents: they are replaced to a character similar to a diamond.

Posts: 7052 · Zenju 4 May 2017, 19:30

Thanks for the beta, but unfortunately the same problem with the accents: they are replaced to a character similar to a diamond. monin, 04 May 2017, 19:23

The problem could also be that the server does not support Unicode. Do you have test login data? Maybe this can be fixed.

Posts: 6 · monin 4 May 2017, 19:35

The server is set to UTF-8. Is that OK? It runs on linux.

Posts: 7052 · Zenju 4 May 2017, 19:37

The server is set to UTF-8. Is that OK? It runs on linux. monin, 04 May 2017, 19:35

Sounds perfect.

Posts: 6 · monin 4 May 2017, 19:39

I have not problem at all with Filezilla.

Posts: 7052 · Zenju 4 May 2017, 19:43

Test login data?

Posts: 6 · monin 4 May 2017, 19:50

No problem! Where can I find your e-mail to send it?

Posts: 7052 · Zenju 4 May 2017, 20:40

At the bottom area of the home page.

Posts: 6 · monin 5 May 2017, 15:54

I want to apologize. Preparing the "test login data" I realized that the problem is at our server with some file names with incorrect accents (they do not show correctly even in our linux server). FreeFileSync is working perfectly!!!

I am trying to figure out what the problem is. Programs like Filezilla can display and copy the correct accents even in the files that are not correctly displayed in the linux server. Any help will be appreciated.