Creating a bucket per user sounds like a bad idea in every blob storage service, not just R2. The pr

Creating a bucket per user sounds like a bad idea in every blob storage service, not just R2. The prefixes are exactly what you should do. Deleting the user data should be easy since you can delete based on a prefix.

R2 does have disaster recovery internally, we don't have user-facing replication (like you can't have two buckets on your account that automatically copy each other)

KarewOP•1/26/24, 3:30 AM

R2 analytics can also be filtered on the prefix. If each user has a UUID, there would be no possibility of overlap

KKarew Creating a bucket per user sounds like a bad idea in every blob storage service,...

Jon Robertson•1/26/24, 3:41 AM

That sounds perfect. I've not used the S3 API properly before. Is the only way to delete the files by prefix by using a lifecycle configuration? I couldn't see anything related to it on the standard delete object requests.

Original message was deleted

Jon Robertson•1/26/24, 4:03 AM

Got it. Thanks for the help!

Erisa•1/26/24, 9:27 AM

Yes but you may need to add some code for it - https://developers.cloudflare.com/workers/reference/how-the-cache-works/

Erisa•1/26/24, 9:28 AM

For image resizing, check here https://developers.cloudflare.com/images/transform-images/transform-via-workers/

Erisa•1/26/24, 9:28 AM

Then on the dashboard, purchase the $0+usage plan of Cloudflare Images (which allows you to use your own storage)

AAnay-208 I'm using cloudflare r2. If I use images, only option I see after buying $0 plan...

Erisa•1/26/24, 9:33 AM

You use the $0 plan to enable Transforms and then use those in a Worker with your R2 as the storage, check this link again https://developers.cloudflare.com/images/transform-images/transform-via-workers/

AAnay-208 So if I'm right, is this how it'll work: - add a custom domain to worker and not...

Erisa•1/26/24, 9:46 AM

Correct

Erisa•1/26/24, 9:47 AM

You may need to implement Cache API within the worker however

Erisa•1/26/24, 9:47 AM

There is an example on that docs page: https://developers.cloudflare.com/images/transform-images/transform-via-workers/#an-example-worker

AAnay-208 Another question, so this is code of my worker function: ```js // Parse reques...

Erisa•1/26/24, 9:55 AM

Yes

AAnay-208 And with that, I'll get charged for $1/100000 images scaled, right?

Erisa•1/26/24, 9:59 AM

You get charged for unique transforms which according to https://developers.cloudflare.com/images/pricing/ is $0.50 per 1K unique transformations per month - 5K transformations included$0.50 per 1K unique transformations per month - 5K transformations included
Unique transformation being same image and same parameters, regardless of cache state

EErisa Its not a hard written limit and more of a "if you go too high, R2 starts to bre...

dave•1/26/24, 5:56 PM

is this documented somewhere? Surprised to see it missing from https://developers.cloudflare.com/r2/reference/limits/

Erisa•1/26/24, 6:10 PM

It's not

Unsmart•1/26/24, 6:10 PM

I'd assume its not there because its not a limit they impose directly its something that number of requests and concurrency can eventually break the system

UUnsmart I'd assume its not there because its not a limit they impose directly its someth...

dave•1/26/24, 7:29 PM

but doing like 1,000 concurrent puts on the same bucket seems like it'll always fail, no?

Unsmart•1/26/24, 7:33 PM

Always maybe right now but in the future not really. It might depend on size of files not sure

Original message was deleted

dave•1/27/24, 1:27 AM

Not yet. You could do a cron Worker that lists new files in a bucket and tracks it, but kinda a pain atm.

Original message was deleted

Hello, I’m Allie!•1/27/24, 12:57 PM

If you run all the uploads through a Worker, then you could just call a function there

Original message was deleted

Hello, I’m Allie!•1/27/24, 1:11 PM

Yes, it would just require a little bit of work

Original message was deleted

dave•1/27/24, 3:03 PM

You will be limited to 100MB on uploads to your Worker if you’re on the free or Pro plan.

Original message was deleted

dave•1/27/24, 3:04 PM

You could look into using gzip on your data, might save you a lot. I know for us, gzip saved us a ton on storing JSON in buckets.

Original message was deleted

dave•1/27/24, 3:05 PM

Ah, what runtime is doing the compression? Is it a Python script or something?

dave•1/27/24, 3:07 PM

If CPU time isn’t an issue, compressing with lzma should have higher compression.

Original message was deleted

dave•1/27/24, 3:08 PM

why not lz4?

dave•1/27/24, 3:09 PM

oh right I am using zstd for btrfs compression already

dave•1/27/24, 3:12 PM

If the download is going through a Worker, you can decompress it on the fly.

Ddave If the download is going through a Worker, you can decompress it on the fly.

dave•1/27/24, 3:13 PM

Or if it’s being viewed in a browser

Original message was deleted

dave•1/27/24, 3:20 PM

What’s annoying is that the etag won’t be the md5 hash of the entire file if you do chunked uploads.

AAnay-208 So do I’ve to use a worker for it then ?

dave•1/27/24, 4:24 PM

your Worker won't be able to accept POST'd 10GB file

AAnay-208 any alternatives

dave•1/27/24, 4:25 PM

you can sign the Content-LengthContent-Length header, but this requires you to know the size that the user will upload.

Creating a bucket per user sounds like a bad idea in every blob storage service, not just R2. The pr

Similar Threads

Similar Threads

Similar Threads