Hello R2, I am trying to track a hang/
Hello R2, I am trying to track a hang/slow down when I download a lot of files very fast. I use R2 to hold data/artifacts for ML training, and before training download a few hundred GB per worker from 16-64 workers.
What I see if things seem to work fine downloading on one worker, but when I ramp it up some files hang or download very slowly (O(kbps)).
I use S5cmd with parallelism, so total download can be ~300 Gbit/s and 300+ class b requests per second. Is that something that's likely to get throttled?
What I see if things seem to work fine downloading on one worker, but when I ramp it up some files hang or download very slowly (O(kbps)).
I use S5cmd with parallelism, so total download can be ~300 Gbit/s and 300+ class b requests per second. Is that something that's likely to get throttled?
Welcome to the official Cloudflare Developers server. Here you can ask for help and stay updated with the latest news
85,436Members
Resources
Recent Announcements
Similar Threads
Was this page helpful?
