Does the donation edition improve google drive performance?

[256shadesofgrey]

Hello,
I want to use FFS for syncing my google drive, and currently, with the free version it takes hours to get through all those small files I have there. So hence the question: would it make the file sync with google drive faster if I was to set up parallel file operations with the donation edition? I know that there are restrictions from google on the number of API calls per second, hence I'm wondering if anyone has tried it.

Another question I have is, will I get updates for the donation edition, and if so, for how long?

[xCSxXenon]

Unlikely. Parallel threads only helps if the transfer speed is artificially limited by Google. You would need to be transferring an extreme amount for Google to decide to throttle you. Hopefully someone who has tried increasing threads for G Drive can chime in, I am only providing an educated guess.
Is it any faster if you do the uploading manually?

Donation receives lifetime updates and unlimited installs

[256shadesofgrey]

Is it any faster if you do the uploading manually?

I have never tried uploading 10k <2KB sized text files at once through their web interface to test it. But it is faster when I do it with the native client while using windows. I would keep using it, but I need to sync on linux, and they don't make a linux version. I tried a bunch of other applications for syncing google drive, and so far the only one that was reasonably fast, was insync, but they demand a yearly subscription to get updates. And a quite expensive one at that. So after trying VGrive, ODrive, Celeste and whatever else is out there, I ended up here, and I would be willing to pay a one time fee if it will be reasonably fast afterwards.

Donation receives lifetime updates and unlimited installs

That is good to know. Thank you.

[LeoW]

Is it any faster if you do the uploading manually?

I have never tried uploading 10k <2KB sized text files at once through their web interface to test it. But it is faster when I do it with the native client while using windows. I would keep using it, but I need to sync on linux, and they don't make a linux version. I tried a bunch of other applications for syncing google drive, and so far the only one that was reasonably fast, was insync, but they demand a yearly subscription to get updates. And a quite expensive one at that. So after trying VGrive, ODrive, Celeste and whatever else is out there, I ended up here, and I would be willing to pay a one time fee if it will be reasonably fast afterwards.

Donation receives lifetime updates and unlimited installs

That is good to know. Thank you.

FYI, there is a native Linux command line app that you use to mount Google Drive directly on Linux. Same command works on Windows with WSL.

It's called google-drive-ocamlfuse.

it's been a while since you have been up here reading this thread So I will hold off on posting detailed instructions on how to use the command until I know that you're going to get the instructions and you're still interested.

I will run a brief performance test comparing free file sync on Linux and windows using this Google Drive Software to the free file sync Google drive functionality that's built into the app. perhaps they operate at similar speed or perhaps one is faster than the other I don't know.

[256shadesofgrey]

Is it any faster if you do the uploading manually?

I have never tried uploading 10k <2KB sized text files at once through their web interface to test it. But it is faster when I do it with the native client while using windows. I would keep using it, but I need to sync on linux, and they don't make a linux version. I tried a bunch of other applications for syncing google drive, and so far the only one that was reasonably fast, was insync, but they demand a yearly subscription to get updates. And a quite expensive one at that. So after trying VGrive, ODrive, Celeste and whatever else is out there, I ended up here, and I would be willing to pay a one time fee if it will be reasonably fast afterwards.

Donation receives lifetime updates and unlimited installs

That is good to know. Thank you.

FYI, there is a native Linux command line app that you use to mount Google Drive directly on Linux. Same command works on Windows with WSL.

It's called google-drive-ocamlfuse.

it's been a while since you have been up here reading this thread So I will hold off on posting detailed instructions on how to use the command until I know that you're going to get the instructions and you're still interested.

I will run a brief performance test comparing free file sync on Linux and windows using this Google Drive Software to the free file sync Google drive functionality that's built into the app. perhaps they operate at similar speed or perhaps one is faster than the other I don't know.

Thank you for the reply. I came across that one too, but I dismissed it because I don't want to mount the drive, since that doesn't create a local copy, so I wouldn't be able to access the files while offline. That's also why I didn't want to use the integrated gnome utility that can do the same. Also command line stuff is a last resort kind of thing for me, I'd rather keep using (just) FFS despite the bad speed. After the initial sync, when I only have to "update" the files, it's not so bad (I made the thread just after I did my first full sync which took like half a day).

But if you have the donate version of FFS and would be willing to test whether using the parallel file operations improves performance, that would be appreciated.

[LeoW]

Well,
Did some performance testing to compare google-drive-ocamlfuse to native freefilesync for google drive. Also tried the Gnome one in Mint but FFS couldn't read the files at all without garbling them up.

Don't bother wasting time with google-drive-ocamlfuse. It is SLOW!!!


With the donation edition (I have) parallel file operations can be increased from one to a higher number for either the local file system or the Google file system or both. I chose 10.

Changing the Google side had NO effect on performance.

Increasing the parallel file operations for the local side made a BIG difference! MUCH better performance.

I went from less than 10 MB/sec to almost 40 MB /sec.

Hope this helps.

[256shadesofgrey]

Well,
Did some performance testing to compare google-drive-ocamlfuse to native freefilesync for google drive. Also tried the Gnome one in Mint but FFS couldn't read the files at all without garbling them up.

Don't bother wasting time with google-drive-ocamlfuse. It is SLOW!!!


With the donation edition (I have) parallel file operations can be increased from one to a higher number for either the local file system or the Google file system or both. I chose 10.

Changing the Google side had NO effect on performance.

Increasing the parallel file operations for the local side made a BIG difference! MUCH better performance.

I went from less than 10 MB/sec to almost 40 MB /sec.

Hope this helps.

Thank you. This is very helpful.

I see you tested with fairly large files (only about 100 files with 7GB total), which basically removes the API request limits set by google to test bandwidth of the rest of the pipeline, since few of those files would be downloaded in less than 1 second. Could you test the parallel operations with like 5-10MB worth of 0-10KB files instead to measure if parallel operations increase the number of API requests that can be made?

Also was your local drive an SSD or an HDD?

[xCSxXenon]

Changing threads on one location versus the other is irrelevant:
"When synchronizing a folder pair FreeFileSync will use the maximum of the number of parallel operations that the two folders support."

https://freefilesync.org/manual.php?topic=performance

[LeoW]

Well,
Did some performance testing to compare google-drive-ocamlfuse to native freefilesync for google drive. Also tried the Gnome one in Mint but FFS couldn't read the files at all without garbling them up.

Don't bother wasting time with google-drive-ocamlfuse. It is SLOW!!!


With the donation edition (I have) parallel file operations can be increased from one to a higher number for either the local file system or the Google file system or both. I chose 10.

Changing the Google side had NO effect on performance.

Increasing the parallel file operations for the local side made a BIG difference! MUCH better performance.

I went from less than 10 MB/sec to almost 40 MB /sec.

Hope this helps.

Thank you. This is very helpful.

I see you tested with fairly large files (only about 100 files with 7GB total), which basically removes the API request limits set by google to test bandwidth of the rest of the pipeline, since few of those files would be downloaded in less than 1 second. Could you test the parallel operations with like 5-10MB worth of 0-10KB files instead to measure if parallel operations increase the number of API requests that can be made?

Also was your local drive an SSD or an HDD?

My Local disk is SSD.

I ran some tests with regular document size files office document type files and my performance basically sucked. I got less than three megabytes per second often less than one.

@xCSxXenon
#xCSxXenon

The developer seems to be correct. It doesn't matter what you set the numbers to it never gets any better than what you get with them set at one each.

What I do not know is if the free version is limited to fewer file operations simultaneously than the donation edition is by default so the maximum number of threads might be different in the free versus the paid I do not know the answer to that you'll have to wait for a response from the developer.

I'm using the donation edition in part because I wanted to donate to a product that I use, and I use a fair amount of open source software and I typically donate to the ones that I use regularly. I was hoping for a performance improvement with the donation edition but apparently, there really isn't any.

I did a complete backup of my Google Drive which consisted of about 54 gigabytes of total files down to an external SSD drive just in case anything happened to my Google Drive files or access to the drive. I started it and let it run when I went to bed and it was done when I woke up in the morning. Now when I run a synchronization to check for changes it runs fairly quickly because most of the data is already local.

ALSO

I have Google drive for Windows installed and I did a second test using my link to Google through the Google software called Google Drive for Windows rather than using the Google Connector inside free file sync to see if there was any difference in level of performance.

The performance was pretty much the same. Copying large amounts of files from Google drive down to your PC is slow.

I tried testing another product on Linux called rclone. It supports about 40 cloud providers and uses I think WebDAV to communicate with Google and thus far I have been unsuccessful getting it to work properly. Google's requirements in order to get it to work is complicated.

If it were possible to talk to somebody at Google technical support they might have a suggestion on the best communication protocol to use the copy files to and from Google Drive only I have no clue how to communicate with anybody real at Google.

Sorry, but It seems as though we are all stuck with mediocre file transfer performance to Google.
There is one more product that I want to download a trial of and see if it can do any better At mapping a drive to Google and getting better performance.

[LeoW]

Changing threads on one location versus the other is irrelevant:
"When synchronizing a folder pair FreeFileSync will use the maximum of the number of parallel operations that the two folders support."

https://freefilesync.org/manual.php?topic=performance

I have the donation edition and this other gentleman has the free edition And we are getting equivalent performance.

I donated not just for the performance but also because I like to support the products that I use in the open source environment although I have to say I am a bit disappointed that the performance is no better using the donation edition than it is with the free edition.

Here is my question for the developer:

If free file sync always uses the maximum number of threads when it's doing an actual copy, What is the purpose behind setting threads to specific numbers in the first place and why do you suggest that performance will be better if the threads are set manually when you know that the software ignores them manually set threads and uses the maximum number of threads available regardless of the settings.

The documentation seems to suggest that you'll get a performance improvement when you're doing a comparison prior to doing a sync and the actual File transfer is the same regardless of whether you have the free edition or the paid edition.

I use the Paid edition because I use the product regularly. I donate to the developer that makes the mail client on my Android cell phone and the mail client on Windows as well.