Revert to text log files!

Posts: 12 · florentine 16 Feb 2020, 02:31

I just installed the latest 10.20 update and found out that the log files have inexplicably been converted to HTML format.

This is an absolutely baffling change. HTML is horribly inefficient to store data nor is it even meant for that purpose in the first place!

Please revert to text logs or if you must have structured data then XML which is at least fit for purpose and separates data and presentation.

Posts: 7212 · Zenju 16 Feb 2020, 08:24

What is the problem with html? How are you using the log files?

Posts: 6 · mrvboring 16 Feb 2020, 11:21

I agree with florentine. These are log files. parsing HTML is difficult compared to plain text, XML, JSON, etc. Also the logs take up more space.

I manage a number of systems that generate different logs from various software. Initially I used to email the log files to myself and manually check them. This got too much, so I wrote a PowerShell script to do it for me and emails me a summary. The summary checks the number of logs is correct, and if found, pulls out errors and includes them. I would need to put in a lot of work to update it for HTML logs.

Maybe you could give the option for either plain text or HTML.

Posts: 10 · harryytm 17 Feb 2020, 12:42

I agree with florentine. These are log files. parsing HTML is difficult compared to plain text, XML, JSON, etc. Also the logs take up more space.

I manage a number of systems that generate different logs from various software. Initially I used to email the log files to myself and manually check them. This got too much, so I wrote a PowerShell script to do it for me and emails me a summary. The summary checks the number of logs is correct, and if found, pulls out errors and includes them. I would need to put in a lot of work to update it for HTML logs.

Maybe you could give the option for either plain text or HTML. mrvboring, 16 Feb 2020, 11:21

Agree on having an option for user to choose which type of log FreeFileSync is generating.

Posts: 1 · Mike D 17 Feb 2020, 22:57

I agree with florentine. HTML is nice but it makes life difficult with sudden unexpected changes like this. The same situation happened the last time log file changes were made. Luckily we test it before putting it into production !

Posts: 68 · TheExpert 18 Feb 2020, 08:17

Hi all,

I agree with your requirement, too. I'm using FreeFileSync with an VBS script that is parsing the log file for error events. Since the log files are in HTML format, this isn't possible anymore. HTML is nice to view but not for automated processes. What about giving the user the choice which log format they want by adding a format selector option?

Thank you

TheExpert

Posts: 29 · bgregory02 18 Feb 2020, 13:31

I also prefer "text" log files. I parse the log files every night for errors. Please make it an option.

Posts: 292 · Gianni1962 18 Feb 2020, 13:52

I agree with making the html log file optional, because I use a vbs script to parse the log file.

Posts: 1 · guycl 18 Feb 2020, 19:21

Yes please : make it optional (eventually in de advanced settings) ...
I was planning on post-processing too .... and a 5.5MB html (19.000 files sync) took forever
to load in my browser (OK, I don't have the most modern computer anymore ..... :-)
Kind regards
Guy

Posts: 10 · randomofamber 20 Feb 2020, 08:09

I would like to get back txt format of log too.
Thank you very much.

By the way.
It will be nice to set limit of log files for each sync job separately. If you work on PC about 8 hour per day and have job that sync each hour and set limit to 30 days, it's 240 log, but if you have concurently another job runnig by RealTimeSync, it make 10000+ logs!

Posts: 12 · florentine 25 Feb 2020, 18:53

I also just noticed that the icon images are being hotlinked directly from the FFS website. It's not even saving a local copy with the installation! This is a complete waste of bandwidth for FFS AND creates privacy implications since every time anybody opens the log in a browser it will ping the FFS servers.

Posts: 2 · daveg 25 Feb 2020, 20:05

I'm in favor of html as a global option. The new format looks nice but I'm doing simple checks with files and dont want a browser to check a text output. Great program.

Posts: 102 · volker01 27 Feb 2020, 07:22

I agree with the others. Parsing html ist complicated.
I check the logfile filtering errors, making a small result-log and so on.
Sending html Logfiles per mail can be blocked from some mail-programs ...
With Windows cmd-tools you cannot easy handle UTF-8 files,
it must first convert in ANSI for "findstr" an so on.
Therefor in Windows you need thirdparty programs, lile gnu-tools for Windows.
Also to convert "html to text" there are some thirdparty tools like:
https://www.nirsoft.net/utils/htmlastext.html.
It's much better Logfiles are txt files!
Please revert to .txt/.log or making an user-option in globalsettings.

Posts: 5 · MauriceH 10 Mar 2020, 15:38

Yes, please make this log file in HTML as an option. And leave log file in plane text!

Posts: 2 · 24scuba 11 Mar 2020, 19:52

Need plain text log files back ASAP! Have to switch back to .19 for now.

Posts: 7212 · Zenju 17 Mar 2020, 18:43

A new global option to select between HTML and plain text has been added for FFS 10.21

Posts: 2 · 24scuba 17 Mar 2020, 19:13

A new global option to select between HTML and plain text has been added for FFS 10.21 Zenju, 17 Mar 2020, 18:43

Thank you!!!!

Posts: 68 · TheExpert 18 Mar 2020, 18:09

@Zenju: Thank you for bringing back the plain text log files.

In the global settings file you can add the format option to this XML tag:

<LogFiles MaxAge="30"/> --> <LogFiles MaxAge="30" Format="Text"/>

It looks like the plain text log files are different from the ones of FFS 10.19 and earlier. In Windows (Server 2012 R2) notepad they aren't shown with line feeds any more. The line feed is only visible by opening the files with Notepad++ or similar editiors.

Hopefully the parsing of these files will work with Visual Basic Script as it was with the ones of FFS 10.19 and earlier.

Posts: 6 · mrvboring 18 Mar 2020, 18:26

It looks like the plain text log files are different from the ones of FFS 10.19 and earlier. In Windows notepad they aren't shown with line feeds any more. The line feed is only visible by opening the files with Notepad++ or similar editiors.

Hopefully the parsing of these files will work with Visual Basic Script as it was with the ones of FFS 10.19 and earlier. TheExpert, 18 Mar 2020, 18:09

Looks like the new logs are using LF for EOL. The previous logs (on Windows at least) used CRLF.

Posts: 29 · bgregory02 19 Mar 2020, 01:44

I'm using programs written in VB6 and and they ARE NOT reading the log file correctly. Looks like each line needs a CRLF not just a LF. I've reverted back to 10.19.

Posts: 68 · TheExpert 20 Mar 2020, 18:53

For me parsing the plain text log files from FFS 10.21 with VB scripts is working again but the parsed lines are listed twice.

Posts: 7212 · Zenju 20 Mar 2020, 19:01

Hm, does it make sense to add CR+LF back for Windows?

I was under the assumption that CR could be considered legacy. It certainly is not used on Linux, deprecated on Mac since OS X, and not needed by (all?) modern text-processing apps (including the tools that ship with Windows).

Are there major usecases that need CR on Windows?

Posts: 68 · TheExpert 20 Mar 2020, 19:10

The default editor of Windows Server 2012 R2 (Notepad.exe) doesn't show the plain text log files correctly if there's no CR. And it looks like VBS is also awaiting a CR for identifying the EOL.

With the default editor of Windows 10 the plain text log files are shown as expected with line feeds.

Posts: 12 · florentine 16 Apr 2020, 06:43

Thanks for adding this back!!!

Posts: 68 · TheExpert 28 Apr 2020, 10:55

Why are the errors logged twice in the plain text log files? It's nice that there's a short list of the error messages when opening and reading the log file manually but when the files are parsed the errors are parsed twice. At the moment I have no idea how to prevent the second parsing.

Posts: 10 · Malbec 28 May 2020, 17:32

@TheExpert
Hello,
I saw you were using a VB script to parse log files. I began to write a parser in Python, but I'm really short of time! Could you, please, send me your VB script?
I'd be very grateful!
Thanks.

Posts: 68 · TheExpert 28 May 2020, 18:24

@TheExpert
Hello,
I saw you were using a VB script to parse log files. I began to write a parser in Python, but I'm really short of time! Could you, please, send me your VB script?
I'd be very grateful!
Thanks. Malbec, 28 May 2020, 17:32

@Malbec
Hello,
here you can get the VB script I'm using and maintaining: viewtopic.php?f=2&t=2095&p=18563#p18563.

Posts: 10 · Malbec 29 May 2020, 10:52

@TheExpert
Thanks a lot!

Posts: 8 · aaronkempf 16 Mar 2022, 19:49

Personally, I kinda like parsing HTML sometimes. Of course, I don't parse form HTML-->TXT files, I parse from HTML-->SQLServer.

If you guys ever need to parse HTML, I *HIGHLY* recommend this function.. But DON'T read the licensing terms, because then you won't use it! Just PRETEND it's open source!

(I'm not a lawyer, please dont' follow my joking recommendation)
https://sourceforge.net/projects/sqldom/