I was able to reproduce this:
- Upload a large file (hundreds of MBs).
- Download 1 MB units in parallel using x20 threads, in a loop, forever.
It takes something like 10 to 20 minutes (using ~100 Mbps) and Google will lock out this file, and further retries just return the same error:
HTTP/1.1 403 Forbidden
Vary: Origin
Vary: X-Origin
Content-Type: application/json; charset=UTF-8
Cache-Control: private, max-age=0
Content-Length: 320
Server: UploadServer
{
"error": {
"errors": [
{
"domain": "usageLimits",
"reason": "quotaExceeded",
"message": "The download quota for this file has been exceeded",
"locationType": "other",
"location": "quota.download"
}
],
"code": 403,
"message": "The download quota for this file has been exceeded"
}
}
This error is undocumented, and only applies to the file in question. Uploading a new file and repeating the same steps above will reproduce it.
So based on that, I don't see a problem with this.
To avoid running into this error, I would suggest:
- Do not use large chunks on I/O heavy drives. The 10 MB default is probably ideal in most cases. This user was using 100 MB chunks (maximum allowed).
- Using large chunks means that more I/O will tend to go to the same file, thus triggering the quota error.
- Do not increase your upload / download threads beyond what your upload / download bandwidth can handle. This user was using 20 / 20 (maximum allowed).
- For example, if you can saturate your upload speed using 2 thread, then there's no need to increase them. It will only be detrimental in terms of performance.
- I can saturate my 35 Mbps uplink with only 2 upload threads to Google Drive.
- Same goes for download threads.
The bad news is, once you do hit the per-file quota limit, it persists on retry. So what happens is, StableBit CloudDrive will retry a number of times and then return an error code to the caller. The caller can then decide whether to retry the read operation again or fail. Windows' I/O manager will retry read operations a number of times before failing.
I'm going to set a hard limit on chunk sizes for Google Drive to 20 MB for all future builds in order to hopefully prevent people from running into this in the future.