Cloudflare Developers•3y ago

what is the latency of R2 storage (without Cache)?

Erisa•1/7/23, 8:57 PM

its per request, so each "chunk" would be a separate read

Erisa•1/7/23, 8:57 PM

so serving video files is more expensive than just a flat downloadable file of the same size

Vvvotekeb what is the latency of R2 storage (without Cache)?

Unsmart•1/7/23, 8:58 PM

It depends. What's the file size, where is your R2 bucket, where is the request coming from.

Unsmart•1/7/23, 8:59 PM

I do happen to track a lot so heres an example map of getting a 25MB file from an EU bucket around the world (no cache)

Matt Silverlock•1/8/23, 3:52 PM

DO for co-ordination (who is in what channel/room), D1 for accounts/user data/refs to content (images, files, etc), R2 for the actual files/uploads/etc

Matt Silverlock•1/8/23, 3:52 PM

(This isn't prescriptive, but a real-time chat app is likely to demand all three, plus potentially Queues for things like email sending, crawling embed tags, etc as you expand)

CubityFirst•1/8/23, 4:58 PM

https://i.cubityfir.st/XubJEUfjPL

CubityFirst•1/8/23, 4:58 PM

I'm still not certain why this bug exists?

CubityFirst•1/8/23, 4:58 PM

Is it known?

CubityFirst•1/8/23, 4:59 PM

I haven't deleted anything from my bucket but Cloudflare still says that it's dropped by 75mb or so

James•1/8/23, 4:59 PM

Stats are just delayed a bit, as far as I know

CubityFirst•1/8/23, 5:01 PM

It'd make sense, the dashboard stats (for unique visitors etc) only update once per hour by default it's just bizarre that for some reason it drops instead of just assuming exactly what it was when it last checked

Walshy•1/8/23, 11:02 PM

Ty let me escalate

Jjlam Hmm the response I got wasn't very useful. I was told that CF offers a range of ...

Erisa•1/9/23, 8:39 AM

Hey sorry about that, it was probably a mistake, I'll have a look for you in a bit

Jjlam Thank you. For now, I'm streaming the videos through Dropbox so at least the sit...

Karew•1/9/23, 8:52 AM

Just FYI, Dropbox is going to clamp down on that too if it gets too much traffic

kev-ac•1/9/23, 9:36 AM

I (maybe?) found a bug within the R2 access token logic. Last evening I've created a R2 access token with 24 hours validity. This morning I was greeted with many 403 Forbidden errors. Looking at the Audit log I've seen that the token was invalidated / has expired at midnight. Maybe this is the expected behavior, but I would expect "Token creation" + 24 hours as the lifetime.

Bildschirmfoto_2023-01-09_um_10.33.43.png

Jjlam Thank you. For now, I'm streaming the videos through Dropbox so at least the sit...

Erisa•1/9/23, 9:44 AM

Hi, I replied to your ticket asking for the affected domains and reproduction URLs - once you get back to us with that on the ticket I can take care of escalating to the right teams

Erisa•1/9/23, 9:45 AM

And you sent it already, awesome. Thanks

kev-ac•1/9/23, 11:09 AM

@Erisa | Support Engineer I don't have a similar issue today but want to prevent the same thing from happening to us in the future. Do you have any tools to make sure our Buckets will not get flagged for this? We are currently not using R2 for video assets in production but seeing these issues makes me nervous about a switch over.

Erisa•1/9/23, 11:11 AM

I'm not sure what to recommend there tbh, the issue is known and involves the automatic abuse detection systems flagging R2 video traffic which needs to be manually overturned. But I don't know of any ways to proactively prevent it from flagging, except for not serving large amounts of video

Erisa•1/9/23, 11:12 AM

Except for the minor probably unhelpful note that this won't happen if you're on an Enterprise plan

EErisa I'm not sure what to recommend there tbh, the issue is known and involves the au...

kev-ac•1/9/23, 11:13 AM

Not that it would be a perfect solution, but would this be a "one time" action? So if it gets flagged and overturned manually, will it prevent it from happening again?

Erisa•1/9/23, 11:13 AM

Yes, it shouldn't happen again after it's happened once on a zone and has been reversed

kev-ac•1/9/23, 11:16 AM

Ok. I think the endpoint does not make a difference? S3 API vs. *.r2.dev vs. Connected Domain?

Erisa•1/9/23, 11:17 AM

S3 API (including presigned URLs) won't be flagged, the issue is when you enable public access

kev-ac•1/9/23, 11:19 AM

Maybe we can work a way out to use a worker to alter our HLS manifest files to include url signing params, but this would create huge overhead as both media segments an manifest files are in the same path prefix.

kev-ac•1/9/23, 11:21 AM

Just to confirm, Connected Domain is generally fine for this use case? I'm still worried about this because it adds things like Cache and these are usually under the Non-HTML limits.

kev-ac•1/9/23, 11:21 AM

Like when data comes from R2 it can be cached on the edge?

Erisa•1/9/23, 11:22 AM

ToS-wise if you're using R2 you're not in a violation of the terms even if you cache it(usual disclaimer applies, I'm not a lawyer). But connecting a domain has been known to flag the automatic detection in some cases even if you use the native Domain Access feature

Erisa•1/9/23, 11:23 AM

I'm not sure if the *.r2.dev*.r2.dev URLs can flag this, I don't think I've seen it before but

I can't guarantee anything and those don't have caching so they may not be ideal for most cases anyway

EErisa ToS-wise if you're using R2 you're not in a violation of the terms even if you c...

kev-ac•1/9/23, 11:23 AM

Thanks for clarifying. A clear informational page in the docs would be great in the future. Reading the ToS for this kind of use-case is confusing for non-lawyers