r2 mass move files (1 million files)
I'm using the aws-sdk/ s3 client to connect to cloudflare.
I assume i was hitting the bucket operations per second.
I see a couple hundred logs of it working, then it pauses for 10 minutes. seems odd might be a different rate limit.
7 Replies
This is where i got the rat elimits from: https://developers.cloudflare.com/r2/platform/limits/
Cloudflare Docs
Limits
1 Bucket management operations include creating, deleting, listing,
and configuring buckets. This limit does not apply to reading or writing objects to a bucket.
2 The object size limit is 5 GiB less than 5 TiB, so 4.995
TiB.
3 The max upload size is 5 MiB less than 5 GiB, so 4.995 GiB.
4 Max upload size applies to uploading a file via one...
See the fineprint there:
Bucket management operations include creating, deleting, listing, and configuring buckets. This limit does not apply to reading or writing objects to a bucket.That limit doesn't apply to object operations (e.g. list, delete).
Ah I guess, i'm moving objects to /trash first instead of deleting.
I want to do like a soft delete.
i assume that my soft delete is copying the image into /trash and that's hitting rate limits.
All object operations (move, delete, get, etc) are not subject to that limit.
oh
That limit is for modifying buckets only.
let me add more tracing/logging and seeing where it's hanging
thanks @Isaac McFadyen !