Sync One Large Drive -> Many Small Drives

Discuss new features and functions
Posts: 4
Joined: 11 Jul 2021

Crusher21

I just built a TrueNAS server. However, since the drives are in a raid-like structure, you can think about them as a single drive.

Previously, I had an assortment of various drives of different sizes (14TB, 12TB, 10TB, 8TB) and each one had a cold mirror backup. Now that I have my TrueNAS server, I can't do the same 1->1 backup that I used to do.

I have 4 primary root folders, but no drive is big enough to hold any single folder by itself.

Is it possible for FreeFileSync to be shown the files to be backed up, the names of the available backup drives, and write the files to the destination that it sees fit? There's not really a way for me to split up the data. I'd have to do a compare against the same folder for each of the drives, but only have certain drives get certain parts of the folder. It's hard to explain, but I'm struggling to find the proper solution here.
The ideal solution would be to build a cold backup of the server with the same hardware configuration, but that isn't possible at the moment and won't be for awhile.

What route should I take to accomplish this?
User avatar
Posts: 3551
Joined: 11 Jun 2019

xCSxXenon

If you have a 'RAID-like' structure and it is treated as a single drive, then what you are having issue with doesn't make sense. It's either a RAID array, a drive pool/storage space, or it's individual drives. If it's either of the first two, you wouldn't have to split it up, so I'm strongly guessing you are just using TrueNAS to share drives over the network, which is a waste, but I digress. If you don't want to actually configure them into a RAID or pool, your only option is to decide how you want to split the data up and do a lot of filter inclusions/exclusions so there is no overlap
Posts: 4
Joined: 11 Jul 2021

Crusher21

I'm not sure I understand what you mean. The problem is this:
FreeNAS RAID array-:----- 8 Backup Drives:
-Folder 1 ------------------ Folder 1 can't fit on drive 1,2,3,4,5,6,7, or 8.
-Folder 2 ------------------ Folder 2 can't fit on drive 1,2,3,4,5,6,7, or 8.
-Folder 3 ------------------ Folder 3 can't fit on drive 1,2,3,4,5,6,7, or 8.

The problem is the backup is NOT a RAID array, it is individual drives of different capacities. With that being said, is it still true that I "wouldn't have to split it up"?

I could probably write a powershell script to do this, but I'd be really surprised if FreeFileSync or no other backup software was able to accomplish this.
User avatar
Posts: 3551
Joined: 11 Jun 2019

xCSxXenon

You don't have a RAID array if the drives are accessible individually, so you should set up a RAID array or a drive pool in FreeNAS. If you keep it as is, you are correct, you would have to split it up, which is less ideal. You would have to create multiple sync pairs and change the filter settings to only include folders that will only add up to a certain capacity that will fit on each drive. An 'easy' way to do this is run the compare and multi-select folders from the quick view in the bottom left to exclude until you get under whatever threshold your drives can store.
Posts: 4
Joined: 11 Jul 2021

Crusher21

The drives in my TrueNAS pool are not individually accessible, I'm running 11 drives in RAIDZ3 (single pool). The backup drives, which are of various different capacities, are individually accessible.

Is your above method still the best approach? Sorry I don't think I explained my setup clearly previously.
User avatar
Posts: 3551
Joined: 11 Jun 2019

xCSxXenon

I understand now! My solution still applies though. I was thinking the backup was a RAID. Yes, you will definitely be stuck with splitting them up. You could use spanned volumes within Windows, but that isn't platform ubiquitous
Posts: 4
Joined: 11 Jul 2021

Crusher21

Spanned drives will be fine until I can get some deals on drives on Black Friday. But, if a single one of those drives fails, will I lose all the data is if it were a RAID 0 configuration?

With your method above, I believe that would work for the first backup. However, when it comes time to resync down the line and there is additional data, how do I know that FreeFileSync will first include already backed up data in its calculation before accounting for new data? Or would I have to do that part manually?
User avatar
Posts: 3551
Joined: 11 Jun 2019

xCSxXenon

You would have to research the risks on spanned volumes.
FFS scans the source and destination to determine what needs to be synced