Cloudflare Developers•3y ago

Why do you ask?

SSid Why do you ask?

Hello, I’m Allie!•10/27/23, 3:55 PM

Just in response to this: https://canary.discord.com/channels/595317990191398933/940663374377783388/1167490608949104730

Hello, I’m Allie!•10/27/23, 3:56 PM

Also because generally when you get a colo in code, it only has the IATA, not the digits, so I was wondering if there was a better name for it

HHello, I’m Allie!Just in response to this: https://canary.discord.com/channels/595317990191398933...

SidOP•10/27/23, 3:56 PM

Oh haha is that so @kian

kian•10/27/23, 3:59 PM

I was referring to the MCPs referenced in https://blog.cloudflare.com/cloudflare-outage-on-june-21-2022/#background, where they operate together as opposed to individual colos/PoPs

Colin•10/28/23, 12:07 AM

we're seeing a decent amount of put: We encountered an internal error. Please try again. (10001)put: We encountered an internal error. Please try again. (10001) when calling R2Bucket.putR2Bucket.put

Colin•10/28/23, 12:08 AM

We're using R2 to store file uploads from users (via a Worker). We're using the request body as a stream, so we're unable to retry (as the stream is already consumed + too big to cache in memory)

Colin•10/28/23, 12:08 AM

I understand intermittent failure is expected in a system like R2, but wondering if anyone has ideas for how to reduce these error rates, or if anyone from CF can take a look

Unsmart•10/28/23, 12:27 AM

What is the error rate you get with that? @Colin

From my own testing it seems the error rate it seems quite low

Colin•10/28/23, 12:27 AM

oh yeah it's definitely quite low, I see it at ~1%

Colin•10/28/23, 12:28 AM

but shows up in logs often enough I'd figure it's worth seeing what's going on

Unsmart•10/28/23, 12:28 AM

1% seems pretty high, whats your upload volume?

UUnsmart 1% seems pretty high, whats your upload volume?

Colin•10/28/23, 12:34 AM

I’d estimate we have ~80k object uploads per week

CColin I’d estimate we have ~80k object uploads per week

Unsmart•10/28/23, 12:39 AM

Hmm R2 should be able to handle 80k uploads per week just fine I do ~40k per day with only a 0.08% failure rate. Do you happen to know the average size of the files and do you know if errors happen more often in certain regions?

Colin•10/28/23, 12:41 AM

Yeah it's definitely not a crazy high volume, so I'm not super sure why

Colin•10/28/23, 12:41 AM

Average size of the file is <=50kb; unfortunately don't have a per-region breakdown

Colin•10/28/23, 12:42 AM

I'm wondering if it's users' connections timing out mid-stream or something

Unsmart•10/28/23, 12:50 AM

That could be possible not sure. 50kb is pretty small and should upload really quick so would be surprising if that many had their internet cut out during that

CColin I understand intermittent failure is expected in a system like R2, but wondering...

SidOP•10/28/23, 1:50 AM

If they’ve started happening over the last couple of days, they may be related to the incident we had. Can you send over your account ID and possibly your bucket’s name?

UUnsmart That could be possible not sure. 50kb is pretty small and should upload really q...

Colin•10/28/23, 1:51 AM

Me too, just don't have any other ideas as to what it could be

SSid If they’ve started happening over the last couple of days, they may be related t...

Colin•10/28/23, 1:51 AM

Unfortunately this is a newer workload (~1.5 weeks) so don't have a huge baseline here. Is it ok if I add you to DM the bucket name?

CColin Unfortunately this is a newer workload (~1.5 weeks) so don't have a huge baselin...

SidOP•10/28/23, 2:02 AM

Yeah sure

etya•10/28/23, 11:26 AM

Hi Team, I need some help with building a custom hls streaming service using R2. I'm aware of the streaming service provided by cloudlfare but i want to build a custom one on R2. I need help to implement streaming of multiple hls files through r2. Do i need to create a presigned url for all the files or is there a better way considering my bucket is not public? Please help!!.

JustinNoel•10/28/23, 12:42 PM

@Sid | R2 @omar

RE: r2

Here's what I ended up doing. When the user changes their profile photo, I update a field in my database called profile_changedprofile_changed.

Then, in my handler if that photo is requested again, I check to see if it's more than 15 minutes since the last profile change. If it is less than 15 minutes, I use a refreshed string to fetch the photo Insert the newest version of the photo. Otherwise, I use whatever is in cache right now. I hope that is the most recent photo.

Sid, how long is a dynamically generated image pulled from r2 lifespan? Do I need to bump this up to an even higher time.? Like a day?

JJustinNoel @Sid | R2 @omar RE: https://discord.com/channels/595317990191398933/9406633743...

SidOP•10/28/23, 4:11 PM

I don't think I understand, R2's objects don't have a lifespan (R2 doesn't cache on it own). I think what you may need to adjust is the cache's duration

alexandernst•10/28/23, 6:11 PM

Is R2 throttling uploads (via API) in any way? I'm experiencing what is looks like throttling. I wrote a simple script that sends files to R2 in batches of 100, and after successfully sending 500 o 600, the network traffic stalls for a minute or so.

BabyShark•10/28/23, 6:16 PM

hi, how can I upload multi images in one time via API?

Aalexandernst Is R2 throttling uploads (via API) in any way? I'm experiencing what is looks li...

Unsmart•10/28/23, 6:36 PM

R2 does not have any throttling. I would probably wager that your ISP might be throttling your data in that case

UUnsmart R2 does not have any throttling. I would probably wager that your ISP might be t...

alexandernst•10/28/23, 6:37 PM

I thought that, but I get exactly the same result when trying to do the work from a hetzner machine in germany (I'm in spain)

mr nooli•10/28/23, 9:15 PM

I'm getting the following error when trying to set the cors policy for r2 - There was a problem setting this bucket's CORS configuration, please try again laterThere was a problem setting this bucket's CORS configuration, please try again later am I being a

or is something down?

mr nooli•10/28/23, 9:41 PM

nvm ^ is working again

tt2468•10/28/23, 11:28 PM

I've got this weird issue with listing objects. The function in the screenshot used to return objects in our bucket, but simply doesn't now. An example object name that exists in our bucket is 1001_x2KasaV5awqZVhPekJvwHU_1696199791682_0.mp41001_x2KasaV5awqZVhPekJvwHU_1696199791682_0.mp4, but providing the function with an account ID of 10011001 somehow offers 0 results.

kian•10/28/23, 11:42 PM

Is the file stored in the bucket root?

Colin•10/28/23, 11:50 PM

hey

has anyone been getting more get: Please look at https://www.cloudflarestatus.com for issues or contact customer support. (10043)get: Please look at https://www.cloudflarestatus.com for issues or contact customer support. (10043)'s than usual lately?

Colin•10/28/23, 11:50 PM

I just started noticing them today

Kkian Is the file stored in the bucket root?

tt2468•10/28/23, 11:54 PM

I think so?

kian•10/28/23, 11:57 PM

Are you running the code with wrangler dev?

tt2468•10/29/23, 12:07 AM

Normal wrangler afaik

tt2468•10/29/23, 12:07 AM

I do know the bucket itself is correct, since the .get().get() method for specific objects works as expected

tt2468•10/29/23, 12:37 AM

Looks like delimiterdelimiter may have been the cause

delta•10/29/23, 9:53 AM

Hello, i am unable to connect R2 bucket to custom domain, Status badge staying at initializing state. but conneting to another domain works well. also worker routing to custom domain is working well. any advise?

gluck•10/29/23, 12:53 PM

Hello, I am connecting to R2 from php, when creating a presigned url the generated url seems fine but browsing to it shows this error:
<Error>
<Code>NoSuchKey</Code>
<Message>The specified key does not exist.</Message>
</Error>

the key and secrets are fine as upload to bucket works fine

alexandernst•10/29/23, 6:37 PM

Is there a way to copy files from one bucket to another without downloading & uploading the files?

Erisa•10/29/23, 7:12 PM

If you had Tiered Cache enabled with DO then thats probably the discrepency. Currently R2 doesn't support tiered cache so the cache is per-colo and there are a lot of those.

Erisa•10/29/23, 7:12 PM

That's something that will hopefully be ironed out in future

Erisa•10/29/23, 7:18 PM

https://dash.cloudflare.com/?to=/:account/:zone/caching/tiered-cache is what I'm referring to, having it enabled would increase cache hit rate on DO but not on R2 due to current limitations.

If you have a Pro plan or higher you can view detailed cache analytics to find out where they're coming from. If not, what does your cache rule look like?

Why do you ask?

Similar Threads