Also he complains that performance is crap in areas now, but maybe thats because he placed 40 lights

Also he complains that performance is crap in areas now, but maybe thats because he placed 40 lights in a very small area, and they are all dynamic now instead of baked. ARGH

IIsaac McFadyen Yeah... I actually prefer the new UI but it's subjective I guess?

Meowcat285OP•6/30/22, 2:16 AM

I love it too

Meowcat285OP•6/30/22, 2:16 AM

Let me see if I can find that screenshot of the monstrosity

MMeowcat285 Also he complains that performance is crap in areas now, but maybe thats because...

Isaac McFadyen•6/30/22, 2:16 AM

Are you targeting devices with an RTX 3000 GPU or greater?

Isaac McFadyen•6/30/22, 2:16 AM

If not you'll want to turn off Lumen, it's a major impact if you don't have the Tensor cores.

Isaac McFadyen•6/30/22, 2:17 AM

Also: Unreal Engine

Bbeelzabub 4 million write requests is only $12 for the initial build but yeah, that's a ve...

Meowcat285OP•6/30/22, 2:17 AM

Again, team with no budget

beelzabub•6/30/22, 2:17 AM

Uncached reads would be $2.16 per user assuming each user is doing uncached 100k / hour reads for 8 hours straight for 20 days of the month.

beelzabub•6/30/22, 2:18 AM

I could be wrong but operationally that fits within the budget

beelzabub•6/30/22, 2:18 AM

5 devs = $10.16 in reads per month

Meowcat285OP•6/30/22, 2:18 AM

Actually, thats not as bad as I thought

Meowcat285OP•6/30/22, 2:19 AM

But its the writes that will be killer

beelzabub•6/30/22, 2:19 AM

I didn't get a write rate per user

beelzabub•6/30/22, 2:19 AM

It might not be that bad

Bbeelzabub I didn't get a write rate per user

Isaac McFadyen•6/30/22, 2:20 AM

Also DDC is peculiar in that if you don't have too many changes to the assets then there won't be as many writes.

Isaac McFadyen•6/30/22, 2:20 AM

So might be reasonable depending on workflow?

Meowcat285OP•6/30/22, 2:21 AM

Problem with calculating write cost, is it not very consistent at all

Meowcat285OP•6/30/22, 2:23 AM

I am gonna have to think on this for a bit

beelzabub•6/30/22, 2:23 AM

Please check my math

$12 for a one-time cost for the initial build
$1.35 / mo for storage assuming you're storing 100 GB for the month.
$10.16 / mo for class B for 5 users assuming each is doing 100k reads per hour, 8 hours a day for 20 work days per month
$0/mo for class A for 5 users assuming each is doing 1k writes per hour, 8 hours a day for 20 work days per month.

Meowcat285OP•6/30/22, 2:24 AM

Actually, that looks good

Meowcat285OP•6/30/22, 2:24 AM

I will still need to think on it for a bit

beelzabub•6/30/22, 2:24 AM

If you do 10k writes/hour then it's $30 for class A

beelzabub•6/30/22, 2:25 AM

But 10k writes per second seems high. That's 2rps of writes per user continuously for the entire hour

Meowcat285OP•6/30/22, 2:25 AM

It probally wont be that high

Meowcat285OP•6/30/22, 2:26 AM

As at most we usally only have 2 users actually doing stuff at a time

beelzabub•6/30/22, 2:26 AM

Then I think you'll likely end up with $0 / mo for writes (excluding the initial build)

Meowcat285OP•6/30/22, 2:27 AM

It really depends on whats going on at the time

Meowcat285OP•6/30/22, 2:28 AM

Anyways, I gotta call it a night, thanks for all your help!

beelzabub•6/30/22, 2:28 AM

good luck

beelzabub•6/30/22, 2:47 AM

@Isaac McFadyen | YYZ01 do you know who's responsible for https://pricing.ceru.dev/r2 ? I'm curious if the pricing configuration accounts for the free limits correctly and the github source doesn't seem to include R2

Cloudflare Workers™ pricing calculator

Estimate your next bill, see the price difference. Unofficial calculator for Cloudflare Workers Unbound, Cloudflare R2, and KV.

Bbeelzabub @Isaac McFadyen | YYZ01 do you know who's responsible for https://pricing.ceru.d...

Isaac McFadyen•6/30/22, 2:50 AM

I do.

Isaac McFadyen•6/30/22, 2:51 AM

It's @toridev (Cerulean without the nickname).

Flare•6/30/22, 4:26 AM

The following features are currently on the roadmap for R2 but not yet implemented.

Public Buckets
Mapping a bucket to your custom domain
Pre-signed URLs are now implemented!

In the meantime, you can use a Worker to replicate the vast majority of functionality such as serving & caching public assets.
Take a look at the Demo Worker example here: https://developers.cloudflare.com/r2/examples/demo-worker/
Also, there's a great community project called Render that's made for R2:
https://github.com/kotx/render

Workers will not be required for making assets publicly available in the future, they are just an interim stop-gap.

beelzabub•6/30/22, 4:38 AM

Yes. It's part of the public buckets work but it'll be one of the last things implemented.

beelzabub•6/30/22, 4:39 AM

The current solution indeed is to have a Worker in between that adds the CORS information you need

beelzabub•6/30/22, 4:43 AM

If anyone writes a unit-tested Worker that evaluates S3 CORS rules and applies them to requests (as JS objects - I can do the XML -> JS piece) that'll make that piece go

faster

Bbeelzabub If anyone writes a unit-tested Worker that evaluates S3 CORS rules and applies t...

hinego•6/30/22, 5:32 AM

request two function of r2

support redirect object
support bind custom domain with prefix

For example:
have two object:
object1/test.txt the content is “test1111”
object2/test2.txt set redirect to object1/test.txt in metadata （the object is 0 bytes file only set matedata）

Then request
https://domain/object1/test.txt
response filename test.txt with content test1111
https://domain/object2/test2.txt
response filename test2.txt with content test1111

and if bind domain2 with prefix：object2

will support use

https://domain2/test2.txt

to request /object2/test2.txt

of course it can be implement with workers but it will cost more

�

𝖆𝖚𝖙𝖎𝖘𝖙•6/30/22, 6:54 AM

?r2-roadmap

Flare•6/30/22, 6:54 AM

The following features are currently on the roadmap for R2 but not yet implemented.

Public Buckets
Mapping a bucket to your custom domain
Pre-signed URLs are now implemented!

In the meantime, you can use a Worker to replicate the vast majority of functionality such as serving & caching public assets.
Take a look at the Demo Worker example here: https://developers.cloudflare.com/r2/examples/demo-worker/
Also, there's a great community project called Render that's made for R2:
https://github.com/kotx/render

Workers will not be required for making assets publicly available in the future, they are just an interim stop-gap.

Bbeelzabub If anyone writes a unit-tested Worker that evaluates S3 CORS rules and applies t...

kian•6/30/22, 7:22 AM

Hhinego request two function of r2 1. support redirect object 2. support bind custom do...

beelzabub•6/30/22, 1:36 PM

I'm aware of the request for symlink or redirect support (this is the 3rd time you're asking). Seems unlikely we'll implement it in any short term. There are several challenges.

That operation doesn't exist at all in S3 which means what does the shape of the REST query look like in a way that remains future-proofed against conflicting with future changes AWS makes?
This operation doesn't exist at all which means how do you make it accessible from tooling?
What is the expectation around behavior if the original file is deleted?
It complicates the metadata layer a bit.

It's probably tractable by hanging it off of CopyObjectCopyObject as a custom copy directive (we have REPLACEREPLACE so probably SYMLINKSYMLINK might work there). That solves 1 and 2 but makes 3 weird since after issuing a CopyObjectCopyObject to create object2/test2.txt --> object1/test.txtobject2/test2.txt --> object1/test.txt, DeleteObjectDeleteObject object1/test.txtobject1/test.txt, the copy "doesn't exist" and is dangling metadata that you'd be billed for that's difficult to cleanup (you'd have to basically try to access that object or encounter it in a list). What happens if you now put a new object1/test.txtobject1/test.txt

is the symlink expected to work? The solution here is probably that such a redirect shows up in ListObjectListObject but always returns a 404 on a GET (but does it respond to a HEAD request? maybe only if there's an option that says "don't resolve symlinks"?)

4 pops up because what happens if you:

PutObjectPutObject object1/test.txtobject1/test.txt
CopyObjectCopyObject object2/test2.txt --> object1/test.txtobject2/test2.txt --> object1/test.txt
CopyObjectCopyObject object1/test.txt --> object2/test2.txtobject1/test.txt --> object2/test2.txt
This isn't even getting into the concept of making cross-bucket symlinks.

The hand wavy concept is easy to comprehend but making sure all the corner cases have defined semantics that don't surprise you in behavior, less so. Object storage systems look like filesystems but they're not.

Hhinego request two function of r2 1. support redirect object 2. support bind custom do...

beelzabub•6/30/22, 1:36 PM

Re #2, that's the cname stuff I mentioned earlier.

hinego•6/30/22, 1:46 PM

Think’s for your reply

Bbeelzabub I'm aware of the request for symlink or redirect support (this is the 3rd time y...

beelzabub•6/30/22, 1:46 PM

Oh and writes are another challenge. If I have B --> AB --> A and I PutObjectPutObject to BB, does that destroy the symlink and create a new BB or does it overwrite AA? Lots of nuance.

hinego•6/30/22, 2:03 PM

I don't think you need to think so much

For example, if exists (a -> b)

When getting the object, just cast the request target to B internally! Return 404 if B does not exist!

When operating B, you do not need to care about A, and the user maintains it by himself

It doesn't matter if A is charged even after B is removed, because it's cheap!

R2 only needs to provide the function of internal jump (and only supports one jump, only from A→B, not A→B→C), and the jump is maintained by the user

hinego•6/30/22, 3:45 PM

For example, if the accessed object metadata exists [x-redirect-object], then change the access target to the value of [x-redirect-object]

For example I set metadata for the object /user3333/test222.txt
x-redirect-object:/object/test.bin

Then when I request /user3333/test222.txt, the target of the request will be redirected (internally) to /object/test.bin, but the returned file name can be test222.txt
If /object/test.bin does not exist, then return 404 normally

The main reason for this is to save storage space if you need to save the same file for multiple users, but they name them differently.

And, if you can support binding prefixes, you can bind a separate domain name to each user, and they can directly access files with their domain name

For example, now two users have saved the same file (test.png), but they are named differently,
User A is named: /static/image/test.png
User B named: /images/background.png
If bind prefix and redirection are supported
Then you can save the file as: /raw/test.png
Create an object for user A: /user_a/static/image/test.png
Create an object for user B: /user_b/images/background.png
All set [x-redirect-object=/raw/test.png] for them
Then bind the domain name [user1.domain.com] and prefix [/user_a] to user A
Bind the domain name [user2.domain.com] and prefix [/user_b] to user B

at this time
User A can use
user1.domain.com/static/image/test.png
User B can use
user2.domain.com/images/background.png
access object

and the object is only saved once

beelzabub•6/30/22, 3:50 PM

My job as a TL is to think through the consequences of decisions. I understand your use case is solved by the requirements you laid out but we don't build features just for 1 customer unless it's a very large ENT contract (& even then, maybe). I need to think through the problem comprehensively so that the feature is designed in a way that doesn't lead to a dead end or have sharp corner cases that surprise customers that aren't you in weird ways.

This is a cost reduction mechanism but only for the customers that can make use of it (& even then, storage is fairly cheap - I don't know how big of a cost reduction this is in practice).

AFAIK no other object storage system has something like this. That gives me pause and makes me want to make sure I've thought through every potential corner case & know how it should work.

Also he complains that performance is crap in areas now, but maybe thats because he placed 40 lights

Similar Threads

Similar Threads

Similar Threads