I'm running into a fair amount of these errors: ✘ [ERROR] Error: put: We encountered an internal

I'm running into a fair amount of these errors:

✘ [ERROR] Error: put: We encountered an internal error. Please try again. (10001)

Ddave I'm running into a fair amount of these errors: ✘ [ERROR] Error: put: We enco...

geomaster•3/15/24, 2:43 PM

Me too. WNAM region. No declared incident. This is a weekly occurence for R2

Ggeomaster Me too. WNAM region. No declared incident. This is a weekly occurence for R2

daveOP•3/15/24, 2:44 PM

bleh

geomaster•3/15/24, 2:44 PM

you get what you pay for

Ggeomaster Me too. WNAM region. No declared incident. This is a weekly occurence for R2

daveOP•3/15/24, 2:45 PM

what kind of files are you uploading?

daveOP•3/15/24, 2:45 PM

Oddly my bucket that I'm uploading PDFs to doesn't seem to error out, but the one I'm uploading JSON to is erroring out.

geomaster•3/15/24, 2:46 PM

Oh, interesting. Yeah, it's happening only with JSON files for me actually

geomaster•3/15/24, 2:46 PM

might be a coincidence though

Ggeomaster Oh, interesting. Yeah, it's happening only with JSON files for me actually

daveOP•3/15/24, 2:47 PM

the thing is my PDF files are being uploaded "slower", since I'm tee'ing to create 3 ReadableStreams and uploading them to two R2 buckets, and Azure.

daveOP•3/15/24, 2:48 PM

so maybe the pressure thingy of a Worker is reducing the odds of an error?

Church Guard•3/15/24, 2:51 PM

howdy all, I have a quick question about large downloads from r2 just failing mid download. It only seems to happen for some users and for those users it happens all the time.

Original message was deleted

Church Guard•3/15/24, 3:10 PM

I guess I'll have to build that functionality out.

Church Guard•3/15/24, 3:14 PM

This caught me off guard tbh, I went months with no issues in our playtest bucket and when we switched over to the prod one (everything is the same except the name), we had issues up wazoo with the exact amount of users.

Church Guard•3/15/24, 3:16 PM

Hm, how do issues like that look?

Church Guard•3/15/24, 3:17 PM

Like is there some documentation of a limit, or it's more of a quirk

Church Guard•3/15/24, 3:17 PM

Zoinks

Church Guard•3/15/24, 3:19 PM

Well, let's see how it all goes then.

Church Guard•3/15/24, 3:21 PM

We're going "live" soon and I guess we'll be scrambling to reply to people to let them know whats up

geomaster•3/15/24, 3:42 PM

To add insult to injury, if you do uploads from a Worker, you can't even retry a failed R2 write in most cases

geomaster•3/15/24, 3:42 PM

Because the stream is gone (if it was coming from the user)

geomaster•3/15/24, 3:43 PM

or you read it into memory and hope you don't hit the poorly-documented 128MB memory limit

geomaster•3/15/24, 3:46 PM

I think we'll be migrating to S3 for our production workloads, this is simply too unreliable. I'm actually having trouble imagining someone running a production service on R2 as it exists today

geomaster•3/15/24, 3:46 PM

unless you're just happy accepting the unreliability

geomaster•3/15/24, 3:46 PM

yep

Ggeomaster To add insult to injury, if you do uploads from a Worker, you can't even retry a...

daveOP•3/15/24, 4:04 PM

we're tee'ing to two R2 buckets at once, so in theory we can write a cron Worker that syncs the two up later... kinda ugly and is now 2x the storage though.

Original message was deleted

daveOP•3/15/24, 4:05 PM

what do you mean?

Julio•3/15/24, 4:07 PM

Hey guys, very new to R2. Any recommendations for uploading big files?. Thanks

Original message was deleted

Julio•3/15/24, 4:07 PM

Could you point me to some docs to learn about it?

JJulio Hey guys, very new to R2. Any recommendations for uploading big files?. Thanks

daveOP•3/15/24, 4:08 PM

I personally prefer splitting the file in the application instead of using multipart files.

Julio•3/15/24, 4:11 PM

Thank you

Original message was deleted

daveOP•3/15/24, 4:35 PM

is this wnam too?

danjenkins•3/15/24, 5:41 PM

So I managed to delete my 1tb of data from r2... except for a few "Ongoing Multipart Upload" which weirdly appear to be 3.94 GB in size! I can't delete these using the gui and I can't delete them using rclone..... any ideas?

Original message was deleted

danjenkins•3/15/24, 6:00 PM

Original message was deleted

danjenkins•3/15/24, 6:01 PM

I hadnt thought about this, thank you!