I'd like to request an optional feature that, when enabled, will check for whether a given folder has greater than a specified number of small files (let's say, for example, 5 files of less than 10 MB, but have either number configurable), automatically place a tarball of the small files in that directory in the target destination rather than copying the files themselves. And then when processing updates in the future, check the contents of the tarball to see if a small file that may get copied there is already present.
The reason is basically that I'd like to be able to use the software to back up a large number of files that I have from many different directories to a mechanical drive, where write performance for many small files can be very poor.
In total I think about 55,000 files totaling about 550 GB (ironic how those numbers lined up), and a large number of them are relatively small files. I suspect that being able to write them as continuous streams, without additional compression, would improve performance over just copying each individual file.
In this case, I don't really mind that the target format will be slightly different than the source, I just care that the items are backed up appropriately and that I can get to them... But due to the large number of small files, and the mechanical drive being used, it will take about 8 hours to complete this backup / sync from a fresh start.
[Feature Request] Tarball for > x small files in folders
- Posts: 4
- Joined: 2 Jan 2026
-
- Posts: 4867
- Joined: 11 Jun 2019
- Posts: 4
- Joined: 2 Jan 2026
I realize this won't really change anything but I would like to point out that some comments in the linked threads about the significant overhead, or significant time involved in compressing or decompressing files wouldn't really be applicable here given that what i'm asking for is more or less just to 'store' the items in a tarball, not to compress them, which is a very different scenario, time wise or CPU wise. But fair enough, I get the point(s).
To your questions in the last thread, though, about how do you determine when to switch to that method; I would think it would be a situation the user would decide for a particular saved configuration file, that they would apply those settings.
I find the "FFS is not a backup utility" comment very strange though, given that the first line on the home page is "FreeFileSync is a folder comparison and synchronization software that creates and manages backup copies of all your important files." But I imagine the back and forth on that has happened x times on this forum already so :shrug:
To your questions in the last thread, though, about how do you determine when to switch to that method; I would think it would be a situation the user would decide for a particular saved configuration file, that they would apply those settings.
I find the "FFS is not a backup utility" comment very strange though, given that the first line on the home page is "FreeFileSync is a folder comparison and synchronization software that creates and manages backup copies of all your important files." But I imagine the back and forth on that has happened x times on this forum already so :shrug:
-
- Site Admin
- Posts: 7506
- Joined: 9 Dec 2007
I'm not getting the scenario: "55,000 files" is nothing, even for a mechanical drive, so your 8 hour wait time is coming almost exclusively from the 550 GB total size.
I wouldn't agree with this sentiment. Personally I think there is little need for "real" backup software, and you can simply sync your important files to some external hard disk and call that a backup. In fact it's the only kind of backup I ever do. In this sense FFS can be considered a backup tool.FFS is not a backup utility
- Posts: 4
- Joined: 2 Jan 2026
I didn't take a screenshot of the graph, but there was a very significant (by that i mean, extending for a significant period of time) dip in the graph when it's working on small files. For most of the graph, the transfer was moving at a fairly consistent speed, and then for some periods it's a fraction of that. The thought occurred to me, that this would allow for those files to be transferred as a continuous stream of data rather than the overhead of a large number of smaller files and the disk stopping and starting in between each one. I would expect somewhere between a 5-15% improvement on the transfer I had. It's not hugely important, just something that occurred to me would be a nice feature if it were possible.
-
- Posts: 4867
- Joined: 11 Jun 2019
"Backup" is just so vague of a term lol
I don't agree that FFS "manages backup copies" personally. It creates them, sure, but it doesn't have any native way to browse them throughout time and restore any particular versions. Again, I can't stress this enough, maybe FFS alone is enough for most people to maintain their backup. You could argue that this alone constitutes FFS as a backup utility.
I use FFS to sync data to my backup drives, but for me, that doesn't make FFS a backup utility in the sense that I would recommend it as one to others first and foremost or standalone. It's a feature-rich copy/paste tool, and an incredible one, but I don't call copy/pasting a backup utility either. I agree it IS a backup "tool" as Zenju said, but I reserve "utility" for things that can stand alone in use. Along with my response in the first link, a fully-fledged backup utility would have snapshots and compressions as features like that OP was inquiring about. I guess FFS either isn't a backup utility or it is and it's a pretty bad one lol
As far as the slowdown OP experienced. It only transfers changes, so it's not like those 55k files will be transferred again. Also, drives don't stop and start between actions, there will be a queue of actions for it to work on. The slowdown is much more likely the overhead on the metadata overhead, but this still wouldn't be solved by syncing into a compressed directory either
I don't agree that FFS "manages backup copies" personally. It creates them, sure, but it doesn't have any native way to browse them throughout time and restore any particular versions. Again, I can't stress this enough, maybe FFS alone is enough for most people to maintain their backup. You could argue that this alone constitutes FFS as a backup utility.
I use FFS to sync data to my backup drives, but for me, that doesn't make FFS a backup utility in the sense that I would recommend it as one to others first and foremost or standalone. It's a feature-rich copy/paste tool, and an incredible one, but I don't call copy/pasting a backup utility either. I agree it IS a backup "tool" as Zenju said, but I reserve "utility" for things that can stand alone in use. Along with my response in the first link, a fully-fledged backup utility would have snapshots and compressions as features like that OP was inquiring about. I guess FFS either isn't a backup utility or it is and it's a pretty bad one lol
As far as the slowdown OP experienced. It only transfers changes, so it's not like those 55k files will be transferred again. Also, drives don't stop and start between actions, there will be a queue of actions for it to work on. The slowdown is much more likely the overhead on the metadata overhead, but this still wouldn't be solved by syncing into a compressed directory either
-
- Posts: 2947
- Joined: 22 Aug 2012
I second xCSxXenon's opinion.
I already worded that in 2018 in viewtopic.php?t=4979#p16776.
In his reaction, Zenju then sort of agreed that the restore capabilities of FFS could be improved/extended.
I am not aware anything has changed since then.
As already stated then, that is not intended as criticism; it is ultimately all about available capacity and making choices.
I already worded that in 2018 in viewtopic.php?t=4979#p16776.
In his reaction, Zenju then sort of agreed that the restore capabilities of FFS could be improved/extended.
I am not aware anything has changed since then.
As already stated then, that is not intended as criticism; it is ultimately all about available capacity and making choices.
- Posts: 4
- Joined: 2 Jan 2026
> The slowdown is much more likely the overhead on the metadata overhead, **but this still wouldn't be solved by syncing into a compressed directory either**
I disagree.
The important distinction here isn't that it's a 'compressed' directory. That's not what I'm asking for.
The claim I'm making is that when copying files from an SSD to a mechanical hard drive, creating an uncompressed tar archive instead of many small files would add extremely little overhead, but the resulting file that would be sent to the mechanical hard drive would be one large file instead of many small files, removing the metadata overhead associated with the hard drive head constantly moving back and forth between the end of a file, to the file system index to write data for the next file, to the data for the next file, etc etc.
I will spend some time working on perhaps a powershell script that can create some sort of test data for x small files totalling some size, and a single large file of a certain size, the transfer times for both, but then also the time for tarring the small files and transferring it to the same destination.
If the feature itself wouldn't be worth the effort, or is veering out of scope for the tool, than I understand, but I want to make sure we're on the same page here on what the ask was
I disagree.
The important distinction here isn't that it's a 'compressed' directory. That's not what I'm asking for.
The claim I'm making is that when copying files from an SSD to a mechanical hard drive, creating an uncompressed tar archive instead of many small files would add extremely little overhead, but the resulting file that would be sent to the mechanical hard drive would be one large file instead of many small files, removing the metadata overhead associated with the hard drive head constantly moving back and forth between the end of a file, to the file system index to write data for the next file, to the data for the next file, etc etc.
I will spend some time working on perhaps a powershell script that can create some sort of test data for x small files totalling some size, and a single large file of a certain size, the transfer times for both, but then also the time for tarring the small files and transferring it to the same destination.
If the feature itself wouldn't be worth the effort, or is veering out of scope for the tool, than I understand, but I want to make sure we're on the same page here on what the ask was
