Hi everyone, I am trying to build a production ready application which serves the same image, upload

Hi everyone, I am trying to build a production ready application which serves the same image, uploaded to r2, to thousands of customers. We are a little bit skeptical on how r2 provides free egress, especially since in our case, the download rate will be thousands of times higher than the storage space occupied

Ddano Gotcha. I see that the console is hitting an undocumented api. Can I use that in...

Erisa•10/14/23, 9:29 PM

You can, there's just no guarantees about it working or staying the same since its undocumented

Oomar4289 Hi everyone, I am trying to build a production ready application which serves th...

Karew•10/15/23, 12:40 AM

Yes, it’s really free egress. Make sure that you add a custom domain to your bucket and configure Cache Rules for it to take full advantage of the cache too

Chris•10/15/23, 3:17 AM

I'm experiencing a bug when attaching a domain to an R2 bucket, which may be specific to specificly related to Punycode/IDN (Internationalized Domain Names).

The DNS records are successfully added and the TLS cert and https endpoint are initialized, but the website serves Error 1014. Additionally, an error message is shown when trying to "reconnect" the R2 custom domain using the "..." menu. Lastly, the DNS entries are not deleted (and the UI provides no options to delete if we delete the R2 custom domain, as shown by orphaned DNS entry bin.føø.com (bin.xn--f-5gaa.com)bin.føø.com (bin.xn--f-5gaa.com).

R2_domain_added142.95KB

f.com_HTTPS190.41KB

omar4289OP•10/15/23, 11:28 AM

hi guys, is there an easy way to serve r2 content through a cdn ?

Oomar4289 hi guys, is there an easy way to serve r2 content through a cdn ?

Hello, I’m Allie!•10/15/23, 11:31 AM

Just add a custom domain

HHello, I’m Allie!Just add a custom domain

omar4289OP•10/15/23, 11:32 AM

Thanks

Original message was deleted

digitalpoint•10/15/23, 5:43 PM

'#' is a special character for URLs (fragment identifier), so I suspect it has something to do with that. In a normal HTTP request, anything after the # is not sent as part of the underlying HTTP request. That being said, it does look like it's possibly a bug in R2. If I have a filename of somefile#123.jpgsomefile#123.jpg and I upload it via R2 dashboard, it shows as being uploaded as somefilesomefile, but accessing it via a URL encoded string (needed since anything after # isn't sent to the server by the browser). So the R2 upload process drops anything after the # (same as a browser would do making the request for it). At which point you would be left with an empty file.

A user could probably upload a carefullly crafted filename to overwrite other filenames. Although if you are allowing users to upload direct to R2 with filenames they choose, they could probably overwrite other files anyway.

If it's user generated content, you really don't want to be uploading them to R2 willy-nilly with the filename users uploaded it as anyway.

digitalpoint•10/15/23, 5:53 PM

Yep... I uploaded a file via the dashboard as a test... somefile#123.jpgsomefile#123.jpg ended up as somefilesomefile in the dashboard. I was too scared to try it with a file that started with

because I didn't want to end up an object that wasn't deletable.

digitalpoint•10/15/23, 5:56 PM

Seems so... it's how browsers handle requests with

in it, so it's not the only place in the world that works that way, but uploading to R2 isn't a normal browser request. Might be something like they are parsing the uploaded R2 object key as a URL path, and if it's a URL path, anything after

would be dropped. It does seem like a bug to me.

digitalpoint•10/15/23, 5:57 PM

Or maybe you just need to URL encode object keys when uploading?

digitalpoint•10/15/23, 5:58 PM

But even if you are supposed to that, the normal R2 dashboard isn't doing it, so...

digitalpoint•10/15/23, 6:05 PM

You might want to try URL encoding it... Looking at the underlying mechanism for uploading an object, the object key is part of the S3 endpoint... so underneath it all, you are sending the request to: PUT https://{account_id}.r2.cloudflarestorage.com/{bucketname}/{object_key}PUT https://{account_id}.r2.cloudflarestorage.com/{bucketname}/{object_key}

So I bet that's the culprit right there... the "filename" being part of the request. And anything after the

isn't valid in an HTTP request.

It might be as simple as URL encoding the filename.

digitalpoint•10/15/23, 6:07 PM

...if that's the case maybe it's just a bug in Cloudflare's dashboard and they need to be URL encoding it themselves too (and they are not).

SShah Has anyone been able to make requests using rust to create bucket etc? I'm havin...

digitalpoint•10/15/23, 6:15 PM

It's not an "official" endpoint, but you can create buckets with Cloudflare API without using S3 API.

public function createR2Bucket($accountId, $bucketName)
{
    $params = [
        'name' => $bucketName,
    ];
    return $this->makeRequest('POST', sprintf('accounts/%s/r2/buckets', $accountId), ['json' => $params]);
}

public function createR2Bucket($accountId, $bucketName)
{
    $params = [
        'name' => $bucketName,
    ];
    return $this->makeRequest('POST', sprintf('accounts/%s/r2/buckets', $accountId), ['json' => $params]);
}

That can change at any time though, and doesn't solve your problem of improperly signed S3 requests. So uploading objects and other operations are going to also fail.

digitalpoint•10/15/23, 6:18 PM

If you mean an endpoint different than https://{account_id}.r2.cloudflarestorage.com/{bucketname}/https://{account_id}.r2.cloudflarestorage.com/{bucketname}/, no. The S3 API does not work on custom public domains for buckets (so for example you can't have a presigned URL within your own domain)

digitalpoint•10/15/23, 6:20 PM

If it works with S3, it should work with R2 (mostly)

digitalpoint•10/15/23, 6:24 PM

Maybe I'm missing something, but postman looks like it's just an abstraction layer on top of the S3 API. If I were you, I'd skip postman and just use the S3 API since that's what it's probably using underneath it all anyway.

Matt•10/16/23, 2:14 AM

Is there any way to associate metadata with a file in R2? It looks like S3 "object tags aren't supported"

Matt•10/16/23, 2:15 AM

Use case is being able to associate a set of files with a "project" and perform bulk actions on them later

omar4289OP•10/16/23, 6:46 AM

If I write data from my worker to r2, will I get billed for the worker execution time while the file is being uploaded ?

bigansh•10/16/23, 7:21 AM

i am uploading images and stuff to my storage. but is there a system that i can follow to delete the stored object if its related party is deleted on the mongodb? like i am uploading an image to r2 & then using the url for the image in the document as a header whoes object is stored in mongo. if that document is deleted in mongo, how do i delete the image in r2? like any guidance?

omar4289OP•10/16/23, 8:10 AM

Mongodb offers change events that you can use to know when entities are deleted, then react upon deletion

omar4289OP•10/16/23, 8:10 AM

I would also suggest that you have cron jobs, every once in a while to delete unreferenced files

omar4289OP•10/16/23, 8:13 AM

You could also use some queues that you trigger from your application when entities that references a file has been deleted/updated

Bbigansh i am uploading images and stuff to my storage. but is there a system that i can ...

Karew•10/16/23, 8:48 AM

You kind of want to structure it the other direction. You want the object in Mongo to be the “truth” or the record of having the file. When you want to delete the record you delete the Mongo object and the file in R2 as part of the same process. The Mongo object has a copy of the R2 key. You should not have any orphaned R2 files if you’re strict about this, and it will also make it so you don’t have to list your R2 files frequently.

Paolo•10/16/23, 9:36 AM

Hi! is the URL signing operation class A or class B for purposes of billing?

Original message was deleted

Paolo•10/16/23, 9:42 AM

I'm going to use it to allow downloads only to those with the URL, would that work?

Original message was deleted

Paolo•10/16/23, 9:46 AM

that's perfect, it's exactly what I want. Thank you

Børge•10/16/23, 10:01 AM

If i want to create an application, where you can upload images and videos. Should i then use R2, or Stream and Images?

The videos dosen't have to be streamed, but i still need thumbnails (the first frame of the video).

kian•10/16/23, 10:04 AM

R2 will be much cheaper than the other options, but it is just object storage - what you store is what users will get. No re-encoding, different qualities, etc.

kian•10/16/23, 10:04 AM

Thumbnails would need to be created by yourself, i.e with ffmpeg

Tomas•10/16/23, 3:04 PM

Hello, I can't remove a custom domain from a bucket. It says "The specified bucket does not exist. (Code: 10006)", which seems incorrect, since I'm literally in that bucket's settings. I mistakenly added my app's subdomain there and thus made it inaccessible. Please help!

TTomas Hello, I can't remove a custom domain from a bucket. It says "The specified buck...

digitalpoint•10/16/23, 3:13 PM

While not a solution to being unable to delete it, Cloudflare really should prevent users from adding a public domain to an R2 bucket that already has a DNS entry. I’ve had multiple users accidentally put the root of their domain as their public domain for an R2 bucket which of course kills their site until they remove it and restore the DNS entry the action overwrote.

Ddigitalpoint While not a solution to being unable to delete it, Cloudflare *really* should pr...

Tomas•10/16/23, 3:14 PM

yes exactly ... I misread the description and thought it was for allowing requests from that domain

killed my app and now I can't undo it

Icestonks•10/16/23, 3:39 PM

Is it possible to stream videos with R2? And if so, how would i go about doing that?

Tomas•10/16/23, 3:52 PM

@Cloudflare (Server Owner) can anybody help, please?

TTomas @Cloudflare (Server Owner) can anybody help, please?

Isaac McFadyen•10/16/23, 4:03 PM

Please don't ping people directly.

Isaac McFadyen•10/16/23, 4:04 PM

Someone will help if they can, but pinging people doesn't get your question answered faster and in fact makes people less likely to help.

Tomas•10/16/23, 4:04 PM

you're right, sorry, I'm just panicking a little here

Ddigitalpoint While not a solution to being unable to delete it, Cloudflare *really* should pr...

Zora•10/16/23, 4:38 PM

To be fair, it does warn you before you confirm the action

Hi everyone, I am trying to build a production ready application which serves the same image, upload

Similar Threads