Cloudflare Developers•4y ago

Yeah. You'd probably want to put cache in front

Meowcat285•6/30/22, 1:54 AM

Now TBH that isnt a issue with R2, any S3 compatible provider would be expensive

Bbeelzabub Yeah. You'd probably want to put cache in front

Meowcat285•6/30/22, 1:55 AM

Well again, lots of writes too, that cant be cached

Meowcat285•6/30/22, 1:55 AM

And during the inital DDC build, I saw upwards of 4 million write requests for my project

Meowcat285•6/30/22, 1:55 AM

Looks like it may be best to keep using a on-prem S3 compatible solution

Meowcat285•6/30/22, 1:57 AM

I would love to put cloudflare in front of the on-prem S3, but I am pretty sure that would be against TOS due to the sheer volume of requests

beelzabubOP•6/30/22, 1:58 AM

Yeah it depends on the TCO. It's generally true that cloud software is more expensive than running your own if all you're comparing is the operational cost against the capital outlay. It starts to get a lot more competitive when you start factoring in what it takes to get to 11 9s of durability + 4 9s of availability

beelzabubOP•6/30/22, 1:58 AM

I would love to put cloudflare in front of the on-prem S3, but I am pretty sure that would be against TOS due to the sheer volume of requests

In front in what way? I don't believe our ToS generally says anything about request volume

Meowcat285•6/30/22, 1:58 AM

In this case, I really dont care about durability, as worst case, the Derived Data Cache can be easilly rebuilt

Bbeelzabub > I would love to put cloudflare in front of the on-prem S3, but I am pretty sur...

Meowcat285•6/30/22, 1:59 AM

I belive it says something about large ammounts of non-html requests

beelzabubOP•6/30/22, 1:59 AM

Are you talking about Cache? That just says non-HTML requests can't be cached without an ENT plan. All that means is that you talk to someone about your use-case and they give you a quote

Meowcat285•6/30/22, 2:00 AM

I cant afford ENT, this is just a small game with literally no budget and about 5 devs

Meowcat285•6/30/22, 2:00 AM

Pro is the best I can afford sadly

Meowcat285•6/30/22, 2:01 AM

So honestly, it may be best for me to stick to on-prem S3, and using a VPN tunnel for the other devs, as I have been doing

Meowcat285•6/30/22, 2:02 AM

And I am working on moving the VPN over to WARP and Cloudflare Access

Meowcat285•6/30/22, 2:02 AM

As with such a small ammount of devs, we can easilly fit in the free plan

Meowcat285•6/30/22, 2:03 AM

Actually

Meowcat285•6/30/22, 2:03 AM

@beelzabub I should ask, is there any limits on how much I traffic I can use with WARP for this kinda stuff on the free plan?

beelzabubOP•6/30/22, 2:06 AM

. @Erisa | Out of office do you know which channel to ask WARP questions?

Meowcat285•6/30/22, 2:06 AM

Its alright, I can just ask on the fourms for that

Meowcat285•6/30/22, 2:07 AM

(Side note: I really do wish there was a more generalized Cloudflare Discord server, for all products)

Isaac McFadyen•6/30/22, 2:11 AM

@meowcat285 I'd recommend against using S3 as a DDC. What I am currently doing is running a Samba share publicly on a VPS (since it's only for a UE5 DDC it doesn't matter that it's exposed).

Isaac McFadyen•6/30/22, 2:12 AM

DDC is a LOT of requests but relatively little bandwidth (400mb for my project currently), which R2 doesn't really suit since R2 optimizes for large bandwidth but few requests.

Meowcat285•6/30/22, 2:13 AM

I had issues with UE4 (planning to move to UE5 soon) crashing when using SMB over the internet due to some sort of read write conflict that I dont understand

beelzabubOP•6/30/22, 2:13 AM

What's your monthly spend budget for storage?

Isaac McFadyen•6/30/22, 2:13 AM

Oh, interesting.

Isaac McFadyen•6/30/22, 2:13 AM

UE5's great FWIW.

Isaac McFadyen•6/30/22, 2:13 AM

There's some issues but it's relatively stable now.

Bbeelzabub What's your monthly spend budget for storage?

Meowcat285•6/30/22, 2:14 AM

Probally $35 at the most (which is not nearly enough)

IIsaac McFadyen There's some issues but it's relatively stable now.

Meowcat285•6/30/22, 2:14 AM

One of the team members is being really stubborn about moving to UE5, as he says the new ui messes with his workflows/mussle memory

Isaac McFadyen•6/30/22, 2:15 AM

Oof

Isaac McFadyen•6/30/22, 2:15 AM

Yeah... I actually prefer the new UI but it's subjective I guess?

beelzabubOP•6/30/22, 2:15 AM

4 million write requests is only $12 for the initial build but yeah, that's a very small budget.

Meowcat285•6/30/22, 2:15 AM

Also he complains that performance is crap in areas now, but maybe thats because he placed 40 lights in a very small area, and they are all dynamic now instead of baked. ARGH

IIsaac McFadyen Yeah... I actually prefer the new UI but it's subjective I guess?

Meowcat285•6/30/22, 2:16 AM

I love it too

Meowcat285•6/30/22, 2:16 AM

Let me see if I can find that screenshot of the monstrosity

MMeowcat285 Also he complains that performance is crap in areas now, but maybe thats because...

Isaac McFadyen•6/30/22, 2:16 AM

Are you targeting devices with an RTX 3000 GPU or greater?

Isaac McFadyen•6/30/22, 2:16 AM

If not you'll want to turn off Lumen, it's a major impact if you don't have the Tensor cores.

Isaac McFadyen•6/30/22, 2:17 AM

Also: Unreal Engine

Bbeelzabub 4 million write requests is only $12 for the initial build but yeah, that's a ve...

Meowcat285•6/30/22, 2:17 AM

Again, team with no budget

beelzabubOP•6/30/22, 2:17 AM

Uncached reads would be $2.16 per user assuming each user is doing uncached 100k / hour reads for 8 hours straight for 20 days of the month.

beelzabubOP•6/30/22, 2:18 AM

I could be wrong but operationally that fits within the budget

beelzabubOP•6/30/22, 2:18 AM

5 devs = $10.16 in reads per month

Meowcat285•6/30/22, 2:18 AM

Actually, thats not as bad as I thought

Meowcat285•6/30/22, 2:19 AM

But its the writes that will be killer

beelzabubOP•6/30/22, 2:19 AM

I didn't get a write rate per user

beelzabubOP•6/30/22, 2:19 AM

It might not be that bad

Bbeelzabub I didn't get a write rate per user

Isaac McFadyen•6/30/22, 2:20 AM

Also DDC is peculiar in that if you don't have too many changes to the assets then there won't be as many writes.

Isaac McFadyen•6/30/22, 2:20 AM

So might be reasonable depending on workflow?

Meowcat285•6/30/22, 2:21 AM

Problem with calculating write cost, is it not very consistent at all

Meowcat285•6/30/22, 2:23 AM

I am gonna have to think on this for a bit

Yeah. You'd probably want to put cache in front

Similar Threads

Similar Threads

Similar Threads