Does generating a presigned url count as an operation? If so which one? A or B?

BBarry_Based_Benson Does generating a presigned url count as an operation? If so which one? A or B?

K

Karew•1/19/24, 7:24 PM

Presigning does not involve the server, it doesn’t cost anything

B

Barry_Based_BensonOP•1/19/24, 7:25 PM

But isn't the file content served by cloudflare? Doesn't it count as a read operation?

B

Barry_Based_BensonOP•1/19/24, 7:25 PM

@Karew

U

Unsmart•1/19/24, 7:27 PM

Generating a presigned url doesnt ever touch cloudflare. But when you actually go to the presigned url it will do whatever operation you told it and count

K

Karew•1/19/24, 7:27 PM

You asked about presigning. Your local S3 client presigns an unsent request. It doesn’t cost anything to presign. It only costs an operation if someone actually makes the request

E

Erisa•1/19/24, 8:57 PM

And then the type of operation charged depends on the type of request you created and used

A

Artefactoid•1/20/24, 12:10 AM

I just had a somewhat, um, weird experience with adding an R2 subscription. When I looked at my Billing Subscriptions, R2 was enabled. But when I attempted to configure it, all I got was the R2 intro funnel page with a big blue button asking me if wanted to add it to my subscriptions. Then I went to cancel the subscription. Only once it was canceled could I then access the actual R2 control panel. But it seems like I can't resubscribe to R2 now. Who should I be asking for help to be allowed to resubscribe to R2? TIA.

K

kian•1/20/24, 1:04 PM

I’ll take a look in 5 - there was an issue where the error is wrong for multi part uploads but I think that was fixed

K

kian•1/20/24, 1:11 PM

Did you test it w/ the example folder in the repo?

K

kian•1/20/24, 1:15 PM

How comes your logs don't have a remaining part? meowthinking

K

kian•1/20/24, 1:15 PM

Ah

K

kian•1/20/24, 1:20 PM

Good point

K

kian•1/20/24, 1:21 PM

timingSafeEqual is the same, where you need to encode the text beforehand and the two buffers have to be the same length but that means you can't rely on

string.length

string.length

string.length

string.length

K

kian•1/20/24, 1:21 PM

> "a".length
1
> "à".length
1
> new TextEncoder().encode("a").byteLength
1
> new TextEncoder().encode("à").byteLength
2

> "a".length
1
> "à".length
1
> new TextEncoder().encode("a").byteLength
1
> new TextEncoder().encode("à").byteLength
2

> "a".length
1
> "à".length
1
> new TextEncoder().encode("a").byteLength
1
> new TextEncoder().encode("à").byteLength
2

> "a".length
1
> "à".length
1
> new TextEncoder().encode("a").byteLength
1
> new TextEncoder().encode("à").byteLength
2

K

kian•1/20/24, 1:21 PM

This lovely stuff

K

kian•1/20/24, 1:23 PM

This is in OP's GitHub bio so it's possible meowlul

K

kian•1/20/24, 1:36 PM

I tested with

a

a and

à

à

à

à & that does seem to be the issue

K

kian•1/20/24, 1:36 PM

that's not very fun

K

kian•1/20/24, 1:48 PM

Yeah, I don't think you need to consider the plan

K

kian•1/20/24, 8:59 PM

You'd need to see the request/response that fails specifically to be able to tell what's going wrong

E

Ercan•1/21/24, 6:23 PM

Hi all!

We have been trying to find a way to how to migrate from Cloudflare images to R2, the reason is cloudflare images is becoming so pricey with a large amount of images. And this will not be scalable. Can someone help us figure out what kind of process we should follow for this. And we are also not happy with the cloudflare images because we did not realize it would be this much vendor lock in.

If someone knows a way for this cloudflare images to R2 migration, that would help us a lot. As we see, there is no API or way to export all image from cloudflare images, and dump it into r2 etc.

Original message was deleted

E

Ercan•1/21/24, 8:29 PM

Thank you Daniel!

Original message was deleted

E

Ercan•1/21/24, 8:30 PM

Thank you as well for providing the example!

E

Ercan•1/21/24, 8:32 PM

And @nora another question, to migrate all images, I will basically need to fetch every single image and put it on r2, and I believe this will be reflected in the billing for "Delivered Image" count for cloudflare images right? I am gonna migrate around 10M+ images.

Original message was deleted

E

Ercan•1/21/24, 8:48 PM

I see thanks, should be good for us.

And a question related to the R2 read-after-write consistency.

Is this also valid for using custom domain for r2 access? We experienced some issues in the past where we put objects and tried to directly read by using the bucket custom domain.

K

kian•1/21/24, 8:49 PM

Custom domains have the default CDN caching behaviour

K

kian•1/21/24, 8:50 PM

So, objects may be cached and writing to R2 won't purge that

Kkian So, objects may be cached and writing to R2 won't purge that

E

Ercan•1/21/24, 8:52 PM

Hm, but if an object from r2 is already not in the cache and just uploaded, and if I try to access it using custom domain, why would it fail if it is already not in the cache, and it should retrieve it from the source right? Maybe I am wrong, can you explain a bit more

K

kian•1/21/24, 8:53 PM

404's are cached but it's impossible to tell why it happened without a repro

Kkian 404's are cached but it's impossible to tell why it happened without a repro

E

Ercan•1/21/24, 8:54 PM

I see, I will try to get the error and time, if I see one more time, so we could investigate maybe.

Kkian 404's are cached but it's impossible to tell why it happened without a repro

E

Ercan•1/21/24, 8:56 PM

I also wanna mention what I am planning for 100% relaibility. I will have a custom image resizer server running that will handle resizing etc, and I will source it to directly r2 without custom domain. But I will connect my own cloudlfare domain to my image resizer, so the results of my image resizer would be cached. And I am hoping I can reach the maximum reliability with this setup.

P

pinky•1/21/24, 9:21 PM

Hello, when I have a private bucket and allow access via Worker with the following code, is it going to load the file into Worker's memory and is there any usage impact compared to accessing a file in a public bucket?

const object = await env.BUCKET.get("key from request");
const headers = new Headers();
object.writeHttpMetadata(headers);
headers.set('etag', object.httpEtag);

return new Response(object.body, { headers });

const object = await env.BUCKET.get("key from request");
const headers = new Headers();
object.writeHttpMetadata(headers);
headers.set('etag', object.httpEtag);

return new Response(object.body, { headers });

const object = await env.BUCKET.get("key from request");
const headers = new Headers();
object.writeHttpMetadata(headers);
headers.set('etag', object.httpEtag);

return new Response(object.body, { headers });

const object = await env.BUCKET.get("key from request");
const headers = new Headers();
object.writeHttpMetadata(headers);
headers.set('etag', object.httpEtag);

return new Response(object.body, { headers });

K

kian•1/21/24, 9:21 PM

Nope

K

kian•1/21/24, 9:22 PM

Public buckets are better if you're not adding auth though

P

pinky•1/21/24, 9:26 PM

Is it also the case with the upload? Because I see various ways to upload a file when using worker API.

// e.g. via curl with --data-binary "@somefile.jpg"
await bucket.put(key, req.body, { httpMetadata: { contentType } });

// from a web page
const formData = await req.formData();
const file = formData.get('file') as File
const body = await file.arrayBuffer();
await bucket.put(file.name, body, { httpMetadata: { contentType: file.type } });

// e.g. via curl with --data-binary "@somefile.jpg"
await bucket.put(key, req.body, { httpMetadata: { contentType } });

// from a web page
const formData = await req.formData();
const file = formData.get('file') as File
const body = await file.arrayBuffer();
await bucket.put(file.name, body, { httpMetadata: { contentType: file.type } });

// e.g. via curl with --data-binary "@somefile.jpg"
await bucket.put(key, req.body, { httpMetadata: { contentType } });

// from a web page
const formData = await req.formData();
const file = formData.get('file') as File
const body = await file.arrayBuffer();
await bucket.put(file.name, body, { httpMetadata: { contentType: file.type } });

// e.g. via curl with --data-binary "@somefile.jpg"
await bucket.put(key, req.body, { httpMetadata: { contentType } });

// from a web page
const formData = await req.formData();
const file = formData.get('file') as File
const body = await file.arrayBuffer();
await bucket.put(file.name, body, { httpMetadata: { contentType: file.type } });

Just to confirm, the first approach is also memory free and almost not blocking the worker but the second approach needs to load everything?

K

kian•1/21/24, 9:38 PM

Yes

K

kian•1/21/24, 9:39 PM

FormData can't be uploaded as-is since it isn't just the file, it has other things

C

chip•1/22/24, 1:20 AM

Hey! Is it possible to slap on

x-amz-expiration

x-amz-expiration

x-amz-expiration

x-amz-expiration in metadata for objects to delete them, without having set up a lifecycle in r2?

H

hoalongnatsu•1/22/24, 1:45 AM

I added my credit card and active R2 subscription, but when I go to R2 page then it shows like the image below. I press "Add R2 subscription to my account account" many times but it won't work

.

H

hoalongnatsu•1/22/24, 1:47 AM

How can I work with this problem?

Cchip Hey! Is it possible to slap on `x-amz-expiration` in metadata for objects to del...

K

kian•1/22/24, 1:49 AM

Nope, but I also don't think S3 supports that either?

Cchip Hey! Is it possible to slap on `x-amz-expiration` in metadata for objects to del...

K

Karew•1/22/24, 1:49 AM

No, only lifecycles are supported at the moment

C

chip•1/22/24, 1:55 AM

Sounds good! Thanks!

K

kian•1/22/24, 3:33 AM

People have had issues with Minio’s JS SDK before when using bucket-scoped tokens so it probably isn’t an issue with your code

C

constant1894•1/22/24, 1:04 PM

Hi Everyone! I'm struggeling to set the content type of a .m3u8 playlist. I can see the correct content type does reach the worker, but once the file is stored on R2 the type shows -- and the file cannot be played. Is it possible that my m3u8 file has some issues?

Cconstant1894 Hi Everyone! I'm struggeling to set the content type of a .m3u8 playlist. I can ...

H

Hello, I’m Allie!•1/22/24, 1:05 PM

Are you adding a content-type with