OK, we've got a windows share with 16m files/folders we are trying to send to a new server. We copied teh whole drive and got that over there, so now we're just trying to get it updated with the latest files.
The scanning is taking forever. 16 hours in, it's at 28M files -- i assume it turns to double, since most files are already there.
I just want it to start copying. If the file doesn't exist -- copy it over. All this scanning is killing me. Just do it!
What can I do?
16 million files/folders...
- Posts: 2
- Joined: 28 Oct 2021
- Posts: 1202
- Joined: 8 May 2006
What type of comparison are you doing?
Time & size, content... ?
Time & size, content... ?
- Posts: 2
- Joined: 28 Oct 2021
Doing the comparison on time & size ... but really what would be good, if it just found files newer than say 2 days, on the source, and then just copied them regardless of the state of the destination.
-
- Posts: 4867
- Joined: 11 Jun 2019
Use robocopy
- Posts: 1202
- Joined: 8 May 2006
How did you do that & about how long did that take?We copied teh whole drive and got that over there
I just ran a scan...
100,000 files in about 15.sec.
If you extrapolate that to 16,000,000, one might think it should take about 40.min (for the scan itself).
That's probably the case.it's at 28M files -- i assume it turns to double, since most files are already there
If you look at the FFS "status line", the = (equal "box"), most likely the largest number, shows files that exist on both sides.
So if you double that number, then add to that the "won't be copied" + "will be copied" + "updated to the right" numbers, you should be close to the (if it ever finished on your end) "### items found" (so somewhere around the 32M number in your case).
Another scan (nonsense scan, just select a left & right to see what it would do)...
850,000 files in 1.min.
This was a two-way sync, 200 K going to the left & 650 K going to the right.
