netflix wouldn't ever use r2, they have their own cache boxes at ISPs and IX's, etc, they need reall

netflix wouldn't ever use r2, they have their own cache boxes at ISPs and IX's, etc, they need really really inexpensive storage

CChaika netflix wouldn't ever use r2, they have their own cache boxes at ISPs and IX's, ...

Alex•3/26/24, 6:48 PM

ah, I thought they used s3

ChaikaOP•3/26/24, 6:48 PM

I think they do use a mix or fallback to s3 for some stuff according to Google, but the vast majority is cache boxes and such

ChaikaOP•3/26/24, 6:49 PM

I haven't seen netflix use anything but IX/ISP cache boxes, iirc not all shows at there though

ChaikaOP•3/26/24, 6:49 PM

R2 would also need replication lol

ChaikaOP•3/26/24, 6:50 PM

eh it's got cached throughput, for their use it might be fine in that aspect

Unsmart•3/26/24, 6:50 PM

Companies of that caliber wont choose cloudflare like that just because its cheap, theres a reason netflix uses AWS its because they have thousands of services they can use.

Unsmart•3/26/24, 6:50 PM

No one does that realistically

AAlex they dont respect privacy, period.

Isaac McFadyen•3/26/24, 6:53 PM

Just an FYI - if they looked at things in customers' buckets then they'd be sued many times over by now, including by enterprise customers and likely some governments too.

IIsaac McFadyen Just an FYI - if they looked at things in customers' buckets then they'd be sued...

rob•3/26/24, 6:54 PM

yeah agreed, i can't see that being whats happening

Original message was deleted

rob•3/26/24, 6:54 PM

hahhaa yea

Alex•3/26/24, 6:58 PM

or use zero-knowledge storage, or encrypt before uploading

IIsaac McFadyen Just an FYI - if they looked at things in customers' buckets then they'd be sued...

Alex•3/26/24, 6:59 PM

yes. but if it's not a zero-knowledge storage, you have absolutely no way to know if the little technician from google or any other storage decides to take a look at your files.

Alex•3/26/24, 7:00 PM

Dropbox too

AAlex yes. but if it's not a zero-knowledge storage, you have absolutely no way to kno...

RAGman•3/26/24, 7:02 PM

would love to implement this for browser use...any tips?

RRAGman would love to implement this for browser use...any tips?

Alex•3/26/24, 7:03 PM

StorJ offers that

Alex•3/26/24, 7:04 PM

storJ is absolutely stellar, except for speed

AAlex yes. but if it's not a zero-knowledge storage, you have absolutely no way to kno...

Isaac McFadyen•3/26/24, 7:06 PM

Yes, except that you can (and should) encrypt stored objects.

Isaac McFadyen•3/26/24, 7:07 PM

R2 does this automatically, as does GCS, but you can provide your own key on GCS.

Original message was deleted

Isaac McFadyen•3/26/24, 7:08 PM

You, if you upload your own keys at GetObject time.

Isaac McFadyen•3/26/24, 7:08 PM

Regardless, this probably isn't a very productive discussion :lul:

Isaac McFadyen•3/26/24, 7:09 PM

Cloudflare definitely does not inspect R2 objects. It's up to you to trust that, but given if it leaked that they'd be immediately sued by enterprise customers using R2 (and likely destroyed as a company) then it's pretty much a given that they don't

Isaac McFadyen•3/26/24, 7:09 PM

Of course, if you don't trust that then you don't need to use R2

ChaikaOP•3/26/24, 7:10 PM

I can tell you without a doubt Cloudflare does not look at your GCS objects. Does that help?

MattIPv4•3/26/24, 7:12 PM

:oki: I hope you own multiple physical buildings then so you can keep redundant local backups of stuff if you're not using online services to store it

MattIPv4•3/26/24, 7:13 PM

Wonderful

MattIPv4•3/26/24, 7:13 PM

Also, given you're so worried about folks inspecting stuff, I should make you aware that you're using Discord currently, and they are absolutely feeding your messages into AI

MattIPv4•3/26/24, 7:14 PM

That it is

MattIPv4•3/26/24, 7:17 PM

Lab coat, ladder and high-vis, the three horsemen of getting into places you shouldn't be

MattIPv4•3/26/24, 7:17 PM

Anyway, I do fear this convo has veered rather off-topic for this channel

Alex•3/26/24, 8:42 PM

I'm not even sure that the problem is rate limiting here. this is the error that we get from the API

Alex•3/26/24, 8:42 PM

Remote Storage: Failed to upload listings/06-2020//**.webp with error: Error executing "PutObject" on "https://*************.eu.r2.cloudflarestorage.com/listings/06-2020/******/**********.webp"; AWS HTTP error: Server error: PUT https://**********.eu.r2.cloudflarestorage.com/listings/06-2020/******/***************.webp resulted in a 500 Internal Server Error response:
InternalErrorWe encountered an internal error. Please (truncated...) InternalError (server): We encountered an internal error. Please try again. - InternalErrorWe encountered an internal error. Please try again. on line# 981

Alex•3/26/24, 8:43 PM

what is it ?

AAlex 350-400 requests per second ?

bun•3/26/24, 8:44 PM

Ok wow this is incredibly slow

Alex•3/26/24, 8:44 PM

so how does one put millions of files in R2?

Alex•3/26/24, 8:46 PM

Can't do that. the application must use one bucket, it's how its designed.

bun•3/26/24, 8:46 PM

That doesnt make any sense for the devs to manage multiple buckets? Same goes for D1. Why are they limiting stuff and telling devs to shard/manage multiple stuff

Karew•3/26/24, 8:47 PM

Cloudflare isn’t saying to shard, it’s just one way to get around the problem currently

Alex•3/26/24, 8:47 PM

then it's a "bug" of R2.

Alex•3/26/24, 8:47 PM

bad implementation

bun•3/26/24, 8:47 PM

Oh so its just temporary software limitation

Karew•3/26/24, 8:48 PM

Can your app be slowed down or use a queue somehow for put requests?

bun•3/26/24, 8:48 PM

That doesn’t explain D1 though, 2 GB is incredibly low for a modern database

Alex•3/26/24, 8:48 PM

well yeah, we'll modify the migration application, but I keep seeing problems with R2. Cloudflare, please work on this product!

Karew•3/26/24, 8:49 PM

D1 is still in beta, but you should ask the D1 people in that channel

KKarew D1 is still in beta, but you should ask the D1 people in that channel

bun•3/26/24, 8:49 PM

From based on their responses they aren’t going to up the limit by a lot. Probably limit up to 10 GB, but its still incredibly low

bun•3/26/24, 8:50 PM

Thats my complaint about cloudflare, they rarely prioritize something? They legit have 100 things in beta

Bbun Thats my complaint about cloudflare, they rarely prioritize something? They legi...

Alex•3/26/24, 8:51 PM

exactly this. they launced 30 new services with mediocre level, instead of fewer services with top-notch level

Alex•3/26/24, 8:52 PM

for what's worth, I also tested cloudflare stream vs bunnyCDN stream, and bunny was faster and more consistent across all over europe.

Karew•3/26/24, 8:52 PM

I really like R2 but you are using a middle-tier file product and paying that price. If you need to critically be able to put thousands of objects per second you should probably shell out for Amazon S3

Alex•3/26/24, 8:53 PM

but its cloudflare. they are supposed to be innovative!

netflix wouldn't ever use r2, they have their own cache boxes at ISPs and IX's, etc, they need reall

Similar Threads

Similar Threads

Similar Threads