Three HDDs destroyed while huge-data sync

[sarana]

Hi, I don't know whether FreeFileSync did this, but I am not able to think of any other reason. I have five HDDs for backup which I have been using several years and just yesterday I wanted to change my backup system. So, big moving.

Copying around 800 GB to an ext. HDD from my laptop's SSD maybe after a third of the amount I received an error that the drive doesn't exist. I was surprised, the drive disappeared from Windows Explorer as well. I plugged it into different USB sockets and nothing, it doesn't turn up. It appears in Disk Managament as unallocated space...

Okay, who cares, I have other ext. HDDs... tried with the other two HDDs and a similar scenario happened. All three HDDs simply "disappeared" while copying files with FreeFileSync, again, hundreds of GB, getting this result perhaps after 10 or more GB. I copied hundreds of GBs to those HDD previously with Total Commander and never had this problem. I never saw a HDD partition to turn unallocated like this.

I can of course recover my data with a file recovery software, regardless the fact that this was an additional backup so I didn't lose any data at all. But if I had there some important data this would be quite an additional hassle, in addition to the fact that I thought FreeFileSync is a reliable syncing software but I was apparently wrong. I am now convinced that it burned three of my HDDs. (I was able to recover the partitions and will try use them again for backup, but with caution and without FreeFileSync).

My question is - is my conclusion correct? And whether it is possible to perhaps reduce the bruteforce of copying so I can still use FreeFileSync and avoid burning HDDs/partitions.

[Zenju]

I don't think this kind of data loss is possible with FFS (which has many safety layers to avoid issues related to "drop of source location", even though it might not always look that way.)

The crash in the dump file you sent is caused by Keyman and can be fixed by updating: https://github.com/keymanapp/keyman/issues/1723

[jdaniele]

Hello sarana,

I've been using FreeFileSync for backups since years, synchronizing thousands of GB and I have never had a problem like you described.
For a fresh copy (no need to sync), i just use the simple Windows interface.
Before saying FreeFileSync is the guilty, it would be interesting to see if a basic windows copy of 800 GB produces the same bad results.
If so, the cause could be at a driver level, maybe interfering with some other software, or at hardware level.
Is your PC overclocked in any way?
Have you any raid driver?

For example a long run in copying file could get the chipset hot and if something is wrong with it, you could get unpredictable results.
I had a problem like that (in a hot summer) many years ago.
The culprit was the chipset on the mainboard.
Anyway, check also HDD cables, power supply (drops in voltage could lead to strange results), DRAMs, the temperature of the HDD during copy, the HDD S.M.A.R.T values if there is any suspicious value ecc.
Last but not least, did you installed any new software?

Bye
Daniele

[sarana]

Hello sarana,

I've been using FreeFileSync for backups since years, synchronizing thousands of GB and I have never had a problem like you described.
For a fresh copy (no need to sync), i just use the simple Windows interface.
Before saying FreeFileSync is the guilty, it would be interesting to see if a basic windows copy of 800 GB produces the same bad results.
If so, the cause could be at a driver level, maybe interfering with some other software, or at hardware level.
Is your PC overclocked in any way?
Have you any raid driver?

For example a long run in copying file could get the chipset hot and if something is wrong with it, you could get unpredictable results.
I had a problem like that (in a hot summer) many years ago.
The culprit was the chipset on the mainboard.
Anyway, check also HDD cables, power supply (drops in voltage could lead to strange results), DRAMs, the temperature of the HDD during copy, the HDD S.M.A.R.T values if there is any suspicious value ecc.
Last but not least, did you installed any new software?

Bye
Daniele

No, I have copied Terabytes of data in Windows through Explorer and especially Total Commander and never had that problem. I recovered the second of the three HDDs (had to use a partitioning software because Windows was not able to make use of the HDD by its disk management alone) and copied the same amount of the same files to the same external HDD without any problem by Total Commander. PC is not overclocked and I have no RAID driver or any such thing. I can copy days in a row never a problem... if it is with Total Commander or Explorer... It happened to three different ext. Harddisks and they were connected in different USB sockets. I believe that new software should not have an effect on copying and moving files. Copying and moving files was possible already in the era of DOS, it is a basic function that should not go wrong.

My sense is that FreeFileSync copies faster than other software (and that I witnessed myself) hence the HDD could be exhausted. This is perhaps total nonsense but for the time being let's keep it as a starting theory based on the fact that other software copy slower and HDD have no failure. I would suggest that FreeFileSync is upgraded with a feature to reduce speed of copying. It would be interesting to see if the HDDs fail with that feature.

Anyway, the first of the three HDDs failed entirely and neither Windows nor commercial partitioning software recognizes it anymore. When FreeFileSync copied the data to that HDD, somewhere in half of the data the harddisk started to squeak and make a sound like a cricket. Again and again, a very weird sound never heard this from an external HDD before. FreeFileSync informed me that it cannot find the HDD and I therefore disconnected and reconnected the HDD in different USB sockets. The HDD could not be identified even in Disk Management. I restarted the laptop with the HDD connected and the laptop (Dell Precision 7730) simply could not start with the HDD in, perhaps trying to identify it unsuccessfully. Even after 15 minutes, the laptop would not start after the restart with the HDD. Unplug the HDD and there you go, laptop is ready to use within a half of a minute. The HDD is thus in total disrepair and useless. The other two HDDs were rendered their partitions "unallocated", one of them is 4TB and the other 10TB (a single HDD). I was able to reformat them and get them back, and the 10 TB HDD's data are now fully recovered by a commercial partitioning software.

Let me know any more information. I would love to find out that FreeFileSync is not the culprit, but for the time being I don't see any other reason. I did install some software after this all happened, and I obviously have a plenty of software in the laptop, e.g., Windows 10. But this certainly should not be an impediment to copying data. 🌞

[jdaniele]

OMG, you completely lost one HDD, that's very bad! :(
I'm so sorry about it. This is a real economic damage!

If the FFS is the culprit, maybe the cause could be found in the quantity of files you asked to handle in one go.
Maybe is something FFS is not able to manage correctly.
Did you find any application or system errors on Event Viewer before the HDD was destroyed?
Something related to stack overflows, access violations, indexes out of range?

I think developers should try to reproduce the fault because if it exists, it is a very risky and bad fault for everyone, me included!

I would start to report here all the info you have about your machine, OS, drivers, number of files to sync and total size, number and types of partitions and so on.

At this point i'm curious to understand what happened...

About the speed, on my machine the external HDD are attached to an USB3 PCIe controller, and I think it has a write cache because the first 2.6 GB of data are usually copied in one second, and that is possible only to RAM. The HDDs are just normal WD Elements, so the transfer rate can't be GB/s.
But the write cache could be also a feature of FFS (i disabled the Windows Write Cache for external HDDs but it continues to exist in some way when i sync files).

To be sure I wanna ask the FFS developers: DOES FFS uses any type of read/write caching?

Thanks and good luck
Daniele

[Zenju]

neither Windows nor commercial partitioning software recognizes it anymore. When FreeFileSync copied the data to that HDD, somewhere in half of the data the harddisk started to squeak and make a sound like a cricket.

It should be fairly obvious from a technical point of view, but just for the record: FFS cannot possibly be responsible for that kind of issue.

[sarana]

...Something related to stack overflows, access violations, indexes out of range?

I am not aware of anything like this. Everything worked with simply sync, the problem was only when I started to copy over 50 or so GB to the USB ext. HDD.

I used FFS to sync around 100 or so GBs before to a USB-C connected ext. drive and that was alright. The problem seems to be related to ordinary USB-3 connected HDDs only.

There is no doubt that what happen is (at least) bizarre, but let's please find the cause of the problem. I'd love to see a function to reduce speed of copying in FFS.

The HDD becomes extremely hot when used with FFS. My subjective perception is that the HDDs don't become so hot when I use Explorer or Total Commander. Could this be something of help?

[sflorg]

I sincerely, and with all due respect don't believe that could happen without underlying conditions / issues with the hard drive. Have you / did you run and diagnostics lately on your hard-drives. What temps?
Suggest two programs: www.hdsentinel.com and https://www.hwinfo.com

[sarana]

Have you / did you run and diagnostics lately on your hard-drives. What temps?
Suggest two programs: www.hdsentinel.com and https://www.hwinfo.com

No diagnostics but it still is strange that the same phenomenon would happen on three HDDs of different brand and size. After the FFS informed me that it cannot find the HDD to copy more files, I wanted to do checking on the HDDs but I don't know how to do that on unallocated partitions. Perhaps your suggestion of software will be useful the next time if this happens again. Windows Disk Management showed their partitions as unallocated and the same I saw in EaseUS Partition Master. I recovered one of the partitions in the 10 TB HDD through EaseUS and created new partition in the 4 TB HDD by the same software. I am not sure whether it would make sense test them now after doing those operations. But I will at one point, thank you. The devastated HDD unfortunately doesn't load in any way at all, so no way to test it.

[sflorg]

It is odd. Though I have to wonder why you used a sync program and not an imaging program for such endeavors. I back-up five rigs every week and would not consider such. This is a very decent sync program for daily changes back-ups and works well when I keep information in-sync with some e.g. NOAA government servers. For all other needs I would suggest programs better suited. Yes you can drive a nail with a brick, but a hammer works better. Look at O&O software.
Take care and glad you recovered.
Heidi-Ann

[xCSxXenon]

What a wild thread... Your HDDs failed because you are slamming them with written data, the most intensive operation for a drive. FFS doesn't copy any faster than through explorer. You should run an extended self-test on those drives, which can take a while, but while run through all the sectors. There are even lower level surface tests you can run as well, but not through Windows afaik. What is 'hot'? Under constant heavy load, drives can easily get to 40C+, 104F, which can feel hot to many people.

[therube]

I'd love to see a function to reduce speed of copying in FFS.

Connect to a USB-2 port ;-).

What type of enclosure?
Are you using the same enclosure for all 3 drives?

Temperatures would be nice to know.
High temperatures, but below thresholds, should be fine (just high).
Likewise, SMART data.

I'm against the idea that FFS caused this.

I'm of the opinion that Windows Explorer is not particularly efficient - in copying large data sets, compared to (Altap) Salamander (or presumably Total Commander).

Not that I know, but I suppose one copy could be "harder" then another copy; i.e., buffer sizes & such, general overall efficiency. In any case, like high temperatures, I'd expect that so long as all is within limits, just because your temp is "higher" or your copy method is "harder", should not cause any issues. Now if you're outside of limits, that's entirely different.

Chipset/controllers sounds like a good thought. Not that I know, particularly, but from rumblings, certain chipsets... have known issues. So maybe...

And in that respect, connecting to USB-2 vs USB-3... as they likely have different chipset/controllers.
Or even differences between been plugged into the front or the rear of a case. (I guess on a laptop, that might ? not apply?)


Crickets, sounds like hardware. Drive going bad.


Had a NAS (single) drive, drop out, wasn't seen across the network.
Tried different things, no dice.
Pulled the drive out, put it into an external USB "caddy" (& with appropriate software, as the drive was Linux based) could then read, & eventually copy the data off the drive (onto a Windows) machine.

So it appears the NAS enclosure had gone bad.
Additionally, the drive itself was very old, & seemingly had at least some bad sectors (or at least there were some, not a lot of, files that were not able to be copied off the drive. But I got 99%...

Then what...? The drive set for a bit, till I got the time to think about it. Figured I'd partition/format it (in Windows) & see I could do with it. Plugged it into that USB caddy. Partition/format, then figured I'd run CHKDSK /F on it, & something with that turned it into a brick. Don't know why, but it's not doing diddly at present. (Again when I get some time, I'll give it another look...)

[sflorg]

Said it once and will say it again: Suggest two programs: www.hdsentinel.com and https://www.hwinfo.com
Even even you have formatted the disc it still has information about its past usage and issues. That information stays on the HD

[sarana]

Said it once and will say it again: Suggest two programs: www.hdsentinel.com and https://www.hwinfo.com
Even even you have formatted the disc it still has information about its past usage and issues. That information stays on the HD

Ran both and both saw the two HDDs as 100 percent perfect and healthy.
Then got a BSOD and uninstalled them. No more BSOD. Having fun. 🌞

Seems like I am the only person in the world who's got harddisks destroyed and turned unallocated while just copying files... I am obviously not enthusiastic to try it again on my two new ext. HDDs, but when I feel brave again I'll try my USB2 ports (excellent idea, by the way, thank you. 😊 )

[sflorg]

Those programs didn't cause any BSOD. Utter nonsense. Both very highly trusted and worldwide used programs. . . Been running for years on dozens of rigs.

[sarana]

Those programs didn't cause any BSOD. Utter nonsense. Both very highly trusted and worldwide used programs. . . Been running for years on dozens of rigs.

I don't dispute that. I would love to have a Windows professional closeby who would resolve my little troubles here and there. Unfortunately, I don't. Most of what I learned about Windows is from YouTube or after trial and error. Anyway, thank you for your time, everybody. 🌞

[xCSxXenon]

I am a technician for a repair shop, mostly working with Windows. Those drives are bad. You can get Gsmartcontrol, and look at reallocated sectors, uncorrectable sectors, failed mechanical starts, pending reallocations, etc. Actually, you aren't the only person to lose drives while copying files. That's how most drives fail, during use. Most of the data recovery I perform in-house usually involves a mechanical drive that has lost it's partition table, leading to 'unallocated', RAW, or missing partitions. In Gsmart, you can also run an extended test that will go over all sectors of the drive, which can take a long time obviously. Note, some drives go inactive during this test, so I usually run a batch script that simply creates/overwrites a txt file on the root of the drive, every 60 seconds. This is plain and simple, failed drives. Nothing to do with Windows or FFS. The only other option I could entertain, is a PSU going bad

[sarana]

You can get Gsmartcontrol, and look at reallocated sectors, uncorrectable sectors, failed mechanical starts, pending reallocations, etc.

Thank you very much. What should I do if a reallocated sector, uncorrectable sector, failed mechanical start, or pending reallocation is found? Is there a software which scans as well as repairs all (or most) of these problems? I remember around twenty years ago there was one, but since then I had never a reason to use it...

[el_sabio_]

I'm tired of reading nonsense. Hard drives fail, in the first post you said you have been using these hard drives for several years. FFS is a very good and indisputable program. Regarding the technician ... better not say what I think.
I have been using FreeFileSync for many years without any problems. Goodbye bots.

[therube]

(Related, Files corrupted when copying to external usb3 hdd.)

[sarana]

@el_sabio_

I'm tired of reading nonsense. Hard drives fail, in the first post you said you have been using these hard drives for several years. FFS is a very good and indisputable program. Regarding the technician ... better not say what I think.
I have been using FreeFileSync for many years without any problems. Goodbye bots.

- I would love to hear from you something meaningful. Anyway, that's why I made this post so that I learn some meaning in the nonsense that apparently happened to me.
- I also considered that the first HDD was old, that's why I tried two others. I find it particularly amazing that three (i.e., all) HDDs that I used failed in a row during the same day when copying with a single app - FFS. Call it coincidence or miracle, but I would love to know why did all three fail with FFS on the same day, and after repairing two of them they didn't fail again when copying with MS Explorer.
- FFS worked perfectly for copying huge amounts of data through USB-C. The HDD in USB-C is a modern, high-quality LaCie Rugged. I plan to get more of these so I can use FFS without limits... 🌈

@therube
- I do not have Microsoft Security Essentials in the machine or at least I am not aware of it. Funny that the forum post seems to be from 2014.

I never said that I am correct. Please, show me where I am wrong. Help me, please.
For the time being it seems that we are all shooting in the dark. But one thing remains, complete surface test of the two HDDs. I am still stuck because I'd like to do it with software which can repeat (potentially discovered) errors immediately, as much as possible. Any recommendation will be most appreciated.

[xCSxXenon]

@sarana Please disregard the other info from therube and sabio. If those values I mentioned are NOT 0, the drive has a mechanical failure. There is no repairing it, you replace it. You already mentioned you don't really think FFS is the culprit, I don't know why they are getting defensive or suggesting other info. The drives are bad, and can probably confirmed through Gsmart. If you run Gsmart, doule-click the quesitonable drive, then go to the attributes tab. That's the info you want. I can look as well if you want to screenshot the page. I work with diagnosing and recovering bad drives all day every day.

[el_sabio_]

Possibly it is the power supply of your pc that causes these incidents. Bye bye.