Multipart uploads
Hi, I did a search but was not able to find a proper answer. What is the proper programmatic way to upload large (multi gig) files? I need large video calls uploaded to R2 for AI purposes. I am guessing that if I use Workers, they will timeout (right?)
13 Replies
Unknown User•3y ago
Message Not Public
Sign In & Join Server To View
Thank you for the detailed answer. I do know what multi part uploads are but from what I read about multipart uploads as it relates to R2, it's not just a traditional multi-part upload. I would have to implement something special to keep track of the parts on the client side right? I want to be able to have a generic endpoint where any file upload can call and just upload a file.
Can I still use a multipart upload for that?
I am curious about this statement: Workers are time-bound on CPU time, so if you can stream your upload, it will not cost you anything regardless of the file size.
Can you point me to some literature about this?
Unknown User•3y ago
Message Not Public
Sign In & Join Server To View
So let's assume that I don't do multi-part (I understand the implications of bad connections and all that but just trying to understand something) and I use workers. I am trying to understand the "Duration". If I upload a 10 GB file, and it takes 20 mins to upload, will the whole upload go through (assuming connection remained stable)?
Unknown User•3y ago
Message Not Public
Sign In & Join Server To View
thanks
Unknown User•3y ago
Message Not Public
Sign In & Join Server To View
@sdnts I linked you in the post. I think the 500MB body limit might be an issue although I am not sure. I guess back to the original (real) ask: How do I get a large file into R2 (programmatically)?
is the S3 API the only real option?
Unknown User•3y ago
Message Not Public
Sign In & Join Server To View
yes from a browser
Unknown User•3y ago
Message Not Public
Sign In & Join Server To View
Ok. That sounds promising.
I'll look into that. Thanks.
Would it be possible to use tus to do this?
Unknown User•3y ago
Message Not Public
Sign In & Join Server To View