Hi! Question: I'm looking for a place to store 2-10T of our model training image / video data that d
Hi! Question: I'm looking for a place to store 2-10T of our model training image / video data that doesn't incur crazy monthly or egress fees, flexible with multiple cloudproviders.
Most importantly, great with large file download performance like .tar file / webdataset or large # of small files if needed.
How cloudflare storage performs in terms of benchmark?
Datapoint: Our model weights live on huggingface and with their cli + hf_transfer we can get 1.2G/s download speed on EC2.
Thanks!
Most importantly, great with large file download performance like .tar file / webdataset or large # of small files if needed.
How cloudflare storage performs in terms of benchmark?
Datapoint: Our model weights live on huggingface and with their cli + hf_transfer we can get 1.2G/s download speed on EC2.
Thanks!



