corrupted files

[Kilea61]

I synchronize my archives with a "mirror" logic.
I realized that some files, for different reasons, are corrupted.
The result is that these corrupted files are synchronized and overwrite the intact copy.
Question: does the program offer a preliminary analysis of the files, warning the user before synchronizing the corrupted ones?

[therube]

How does one go about determining what is a "corrupted" file?

I have a text file that I can open in my text editor, & yet, that very same file will not play in my media player.
So that file, is not corrupt to my text editor, yet it is "corrupt" to my media player.

Is the file corrupt or not?

You could check file "headers" with a utility like file (command), but that does what - it checks the headers. And if the header is "correct", yet the rest of the file is otherwise corrupted, who is to know?

So in what manner is "corruption" to be determined?

You can compares files for "exactness", but I don't know how you can determine "corruption".

(Obviously, specific file types, you can open in specific programs, & that might point out corruption - or not.)

[Kilea61]

I agree. But I was just thinking about a control (even if I can guess how heavy it is) that, according to the file extension, would verify with a fast simulation in background, the correctness or the corruption.
The fact is that in this situation I do nothing but propagate the corrupted files.
In short, I understand your considerations but, you have to admit, it's a nice limitation to synchronized and/or security copies.
I used to be proud of my security and backup policies and now I'm really frustrated.


Translated with www.DeepL.com/Translator (free version)

[therube]

Yes, you could take a .txt file & search for "binary" data within it - which might point to corruption.

Or you could take a video file & do something like:
ffmpeg -i "%~1" -v error -f null - 2>&1 | tee "%~1_ERROR.log"
which might point out "issues" with a file. (While at the same time, being very resource & time consuming).

Imagine you have a bad drive. All drives are bad - at some point.
Today, as far as you know, it is working just fine.
Then you go to open a file, & you notice it is not right - the file is corrupted.
And you say to yourself, hmm... that's strange, wonder what happened?
A week later you go to open another file, & you notice it too is not right - the file is corrupted.
And then, it dawns on you, "oh, I had another like that last week... I wonder...?".
And then as you specifically look at other files, you're finding more & more & more of them are corrupted.
And with that, you utter (some four letter word) & shout out, "I bet my drive is going bad!".

But until you have realized that, all during that past week, & for periods of unknown time before, you had been coping files back & forth & forth & back - all the time expecting them to be "correct", when in fact, they hadn't been, for quite some time.

Maybe with rotating backups & versioned backups, you might end up happening to have the same file in both corrupted & good states. How far you go with that (& various other checks) will all depend on the importance of your data.

[xCSxXenon]

This is all way too much work for not having a robust backup method. Turn on versioning, and have multiple backup

[Kilea61]

I actually have a pretty thorough backup policy.
I periodically back up documents to a hard drive.
Not only that, I also do mirror "synchronization" to another hard drive. And that's not all: I also synchronize documents on NAS and, most importantly, on "Google drive".
The problem, as I explained, is at the origin. Part of the original files are corrupted.
And this despite the fact that the hard disk that contains them is "perfect" (continuously monitored with "Hard Disk Sentinel").
Actually, I have not yet understood why some files (mp3, flac, jpeg, raw-nef and some pdf and executable files) have become corrupted.
I finish: I have AVG suite that monitors also ransonware problems and it doesn't detect anything.
So, excuse me but I insist: it would be definitely an important implementation if a backup or synchronization program could detect corrupted files at the beginning in order to avoid that they are "replicated" (overwriting healthy files) in the backup archives.
Obviously I will continue to investigate why some files get corrupted but unfortunately my "diligent" backup policy, paradoxically, has done damage instead of offering me solid preservation guarantees.

Translated with www.DeepL.com/Translator (free version)

[xCSxXenon]

If you have a proper backup solution, just retrieve that last good copy. FFS is only for syncing locations, I would like to avoid feature creep. AVG is also a waste of resources anyway. Just because a drive reports healthy, doesn't mean it isn't bad either. If you aren't versioning your backups, there is a hole in your disaster recovery strategy. Right now, a ransomware or encryption attack would render all of your data, including your backups, useless. Turn on FileHistory in Windows and utilize restore points. You can access previous versions from right on your local machine

[Kilea61]

Thank you