I have a folder that contains over 800 folders, which in turn contain a combined total of over 5.4 million files. New files are regularly added to various folders, but there is no set schedule as to how many or which folders will be added to.
I'd like to maintain a backup of this master folder by periodically manually connecting a different drive, running an sync operation, and then disconnecting the backup drive. Files are never moved or deleted from the source folders, so there is really no need for a Mirror operation, just an Update.
Problem: With so many files, it takes over 2 hours just to scan.
There must be at least a few methods for speeding this up, given all I want to do is add to the backup any new files that have been created in the source folders.
What would be the most efficient way to maintain such a backup?
A way to avoid scanning unchanged folders?
- Posts: 8
- Joined: 25 Nov 2018
- Posts: 8
- Joined: 25 Nov 2018
Anything on this?
-
- Site Admin
- Posts: 7052
- Joined: 9 Dec 2007
FFS's new Google Drive implementation satisfies your requirements: Google Drive supports reporting of change deltas, which allows for folder traversals to be buffered locally and finish instantaneously, no matter the file count.
- Posts: 8
- Joined: 25 Nov 2018
I'm not sure I understand how that would work for my scenario. What would the process be for utilizing this feature to accomplish the task of performing an update sync from one drive to another, ignoring folders that have not changed?
