Hey guys. Is there any difference in the following two examples? Is one preferred over the other? no

Hey guys. Is there any difference in the following two examples? Is one preferred over the other?
note: file here is a File object.

await bucket.put(key, file.stream(), { httpMetadata: { contentType: file.type }});

await bucket.put(key, file.stream(), { httpMetadata: { contentType: file.type }});

await bucket.put(key, file.stream(), { httpMetadata: { contentType: file.type }});

await bucket.put(key, file.stream(), { httpMetadata: { contentType: file.type }});

await bucket.put(key, await file.arrayBuffer(), { httpMetadata: { contentType: file.type }});

await bucket.put(key, await file.arrayBuffer(), { httpMetadata: { contentType: file.type }});

await bucket.put(key, await file.arrayBuffer(), { httpMetadata: { contentType: file.type }});

await bucket.put(key, await file.arrayBuffer(), { httpMetadata: { contentType: file.type }});

HHyper Hey guys. Is there any difference in the following two examples? Is one preferre...

geomaster•3/16/24, 5:30 PM

The second will read the full file to memory, then upload it

The first will stream the file and upload it chunk by chunk

The first one is usually preferred, since you would have the best latency, and more importantly the second example will hit Workers memory limit if the file is too big or if there are a lot of concurrent uploads.

kian•3/16/24, 5:31 PM

Since it’s a File object I assume it’s from FormData so it was already buffered into memory

Kkian Since it’s a File object I assume it’s from FormData so it was already buffered ...

HyperOP•3/16/24, 5:45 PM

yep exactly this

HyperOP•3/16/24, 5:45 PM

which is why i’m wondering if there’s any difference

Isaac McFadyen•3/16/24, 6:26 PM

I wonder if the second one still chunks it or if it's all a one-shot upload?

Isaac McFadyen•3/16/24, 6:26 PM

If it doesn't chunk it then the first might be better even if it's already in memory

HyperOP•3/16/24, 7:21 PM

this snippet is from the r2 docs. When in local mode, are files actually uploaded or does r2 simulate uploading files but theyre not actually uploaded and thus not retrievable? Im having trouble displaying images that i uploaded locally (and then fetched from r2) so i was debugging my code but i suspect the issue is not with my code

HHyper this snippet is from the r2 docs. When in local mode, are files actually uploade...

Hello, I’m Allie!•3/16/24, 7:39 PM

They are stored in the

.wrangler

.wrangler

directory by default

HHello, I’m Allie!They are stored in the `.wrangler` directory by default

HyperOP•3/16/24, 7:47 PM

I understand that, but are they actual files or just blobs simulating the uploaded files

HHyper I understand that, but are they *actual* files or just blobs simulating the uplo...

Hello, I’m Allie!•3/16/24, 7:49 PM

They are actual files. They don't have filenames/metadata, but they do exist as actual files

HyperOP•3/16/24, 7:56 PM

Perfect. Thanks for the response!

HyperOP•3/16/24, 8:19 PM

lol just found this line of code funny

await bucket.delete((await bucket.list()).objects.map((obj) => obj.key));

await bucket.delete((await bucket.list()).objects.map((obj) => obj.key));

Phillip•3/17/24, 9:21 AM

Wondering if anyone can shed some light on an issue im facing with r2.dev access

https://pub-6bb82e4d82e847c89a7edea5d610dd37.r2.dev/65f16b9329b29ca8ddde7980/29one7.png

https://pub-6bb82e4d82e847c89a7edea5d610dd37.r2.dev/65f16b9329b29ca8ddde7980/29one7.png

loads in discord, but no where else for me
https://pub-6bb82e4d82e847c89a7edea5d610dd37.r2.dev/65f16b9329b29ca8ddde7980/29one7.png

Screenshot_2024-03-17_at_10.55.49_am.png

Phillip•3/17/24, 9:34 AM

Ok thanks i was just testing but i might hook up a custom domain. Appreciate the information

Yakir Reznik•3/17/24, 12:00 PM

Hello. I have a frontend app that uses a PHP backend. I have image files that only authenticated users should be able to view. Authentication is done via cookies and headers against the PHP server.
The frontend app gets data from PHP including the urls of said images and then these urls are embedded using <img> tags.
Currently i have an .htaccess file that re-routes requests to these images to a php function that checks if the user is authenticaed and authorized and if so serves the image from disk.
I would like to move all the images to an R2 instance.
I would like to issue a cookie for the user when they log in and i want the users to use that cookie to get the images from R2 directly.
Is that at all possible?

Yakir Reznik•3/17/24, 12:15 PM

@Space - Ping in replies I read about signed URLs.
I would like to avoid re-calculating (pre-signing) the image urls each time they are served.
Is there a way I can issue a cookie or an access token that is good for say 24 hours and then the frontend app can use this cookie or token on all of it resquests to R2?

Yakir Reznik•3/17/24, 12:18 PM

Hmm interesting... i would still need to cache it my PHP server to not re-sign it for as long as its valid

Isaac McFadyen•3/17/24, 2:40 PM

Keep in mind that pre-signing doesn't do any network IO and is simply computing a URL based on input parameters - so it's fairly lightweight to compute.

IIsaac McFadyen Keep in mind that pre-signing doesn't do any network IO and is simply computing ...

Yakir Reznik•3/17/24, 2:40 PM

Thank you. That's good to know

Original message was deleted

Chaika•3/17/24, 6:20 PM

R2 Custom Domains are read-only and only allow you to get files you know the names/keys of. There's no listing or uploading

NNoel it doesn't work with `r2.cloudflarestorage.com` either, i followed <https://deve...

conqr ᯅ•3/17/24, 7:32 PM

I'm having an issue when using the example in https://developers.cloudflare.com/r2/api/s3/presigned-urls/#presigned-url-alternative-with-workers

Do I need to specify the accessKeyId and secretAccessKey when signing the request?

The presign gives the error "Invalid Argument: Credential access key has length 0, should be 32"

Cloudflare Docs

Presigned URLs · Cloudflare R2 docs

Presigned URLs are an S3 concept for sharing direct access to your bucket without revealing your token secret. A presigned URL authorizes anyone with …

conqr ᯅ•3/17/24, 7:36 PM

never mind, I see, its an example where they ommitted actual values

conqr ᯅ•3/17/24, 7:36 PM

that's incredibly misleading

Idriss•3/18/24, 12:19 AM

Hi everyone, I'm new to R2. I want to know if a domain connection is required to make a bucket public? Do I need to buy a new domain for this? (I don't want to use R2.dev subdomain as it is ratelimited).
Thank you!

IIdriss Hi everyone, I'm new to R2. I want to know if a domain connection is required to...

Idriss•3/18/24, 12:27 AM

(Sorry I know for sure that this question has been answered before but I couldn't find it for the moment)

IIdriss Hi everyone, I'm new to R2. I want to know if a domain connection is required to...

Idriss•3/18/24, 12:32 AM

Found the answer which is YES, thank you!

Idriss•3/18/24, 12:42 AM

I have already a domain, and apparently I can use a subdomain for making my bucket public, is there any resource that could help me set this up? because I accidently connected my main domain and one minute later I received an email complaining that my app crashed :blob_sweat:. So, I don't want to repeat the same mistake, is creating a subdomain and connecting it to a bucket documented somewhere?

IIdriss I have already a domain, and apparently I can use a subdomain for making my buck...

Chaika•3/18/24, 1:26 AM

You just enter the subdomain/hostname under Connect Domain

CChaika You just enter the subdomain/hostname under Connect Domain

Idriss•3/18/24, 1:27 AM

okay thank you

Idriss•3/18/24, 2:02 AM

is this normal, I have nothing on my bucket yet and already the number of A and B class operations increases randomly?

Unsmart•3/18/24, 2:04 AM

viewing the dashboard makes requests to the bucket

Idriss•3/18/24, 2:11 AM

Ah, okay. Even though the numbers seem high because I just created the bucket, it's good to know anyway. Thanks!

crux•3/18/24, 8:30 AM

Hi.
Looks like Sippy cannot work with files whose names contain non-Latin characters.

Two examples of links copied from the GCS console and translated into links for R2 with a domain bound to a bucket.

GOOD
GCS:
https://storage.googleapis.com/qq-prod/game/f5c34964-cf4e-491f-aa64-9722b3b1871f/DetectiveParty_Afisha_Ciga-ihOqGbapPi7f-648395.jpg

R2:
https://r2.qq.land/game/f5c34964-cf4e-491f-aa64-9722b3b1871f/DetectiveParty_Afisha_Ciga-ihOqGbapPi7f-648395.jpg

BAD
GCS:
https://storage.googleapis.com/qq-prod/game/03bd26d3-ecb6-4a89-b130-b771e6d1dc87/QQ_%25D0%25B4%25D0%25BE%25D1%2581%25D1%2582%25D0%25B0%25D1%2582%25D1%258C%2520%25D0%25BD%25D0%25BE%25D0%25B6%25D0%25B8%2520%25D0%25BA%25D0%25BE%25D0%25BF%25D0%25B8%25D1%258F1-cAZHN4ZClv7y-115610.jpg

R2:
https://r2.qq.land/game/03bd26d3-ecb6-4a89-b130-b771e6d1dc87/QQ_%25D0%25B4%25D0%25BE%25D1%2581%25D1%2582%25D0%25B0%25D1%2582%25D1%258C%2520%25D0%25BD%25D0%25BE%25D0%25B6%25D0%25B8%2520%25D0%25BA%25D0%25BE%25D0%25BF%25D0%25B8%25D1%258F1-cAZHN4ZClv7y-115610.jpg

Ccrux Hi. Looks like Sippy cannot work with files whose names contain non-Latin charac...

Karew•3/18/24, 9:00 AM

I'd suggest migrating directly instead, using something like Rclone https://rclone.org

Karew•3/18/24, 9:01 AM

Although I would consider filenames like this one to be a smell https://r2.qq.land/game/03bd26d3-ecb6-4a89-b130-b771e6d1dc87/QQ_%D0%B4%D0%BE%D1%81%D1%82%D0%B0%D1%82%D1%8C%20%D0%BD%D0%BE%D0%B6%D0%B8%20%D0%BA%D0%BE%D0%BF%D0%B8%D1%8F1-cAZHN4ZClv7y-115610.jpg

KKarew Although I would consider filenames like this one to be a smell <https://r2.qq.l...

crux•3/18/24, 9:03 AM

This is User Generated Content and this does not mean that files with such names should not be processed.

Karew•3/18/24, 9:04 AM

You should not be letting users name the file keys you store in S3-like services, you should ideally track that filename seperately somehow and store the file under a predictable key

crux•3/18/24, 9:06 AM

However, this is a bug. Files with such names are possible, do not violate any standards and work with direct distributions from other S3-like services. In addition, I want to serve files directly from R2/S3, and not waste resources on retrieving a separately stored name.

Isaac McFadyen•3/18/24, 12:13 PM

One of the issues with letting users name their own files is that they could upload a file with / I think?

Isaac McFadyen•3/18/24, 12:13 PM

Then suddenly they've created a new "directory" in your R2 bucket.

Hello, I’m Allie!•3/18/24, 12:19 PM

Good think you can't do directory traversal ala

../

../

Ccrux Hi. Looks like Sippy cannot work with files whose names contain non-Latin charac...

crux•3/18/24, 1:38 PM

This is not about how to “correctly” host user files. The point is that Sippy is expected to correctly migrate a file with URL-coded characters in the name.

dsky•3/18/24, 2:12 PM

Hi, I'm new to R2 and have read the docs and tried to use the Postman collection. I am not using S3 but simply want to push files via an API to R2. I do not find a proper APi documentation on how to send proper requests. Can anybody point me to what I might be missing please?

Ddsky Hi, I'm new to R2 and have read the docs and tried to use the Postman collection...

Chaika•3/18/24, 2:13 PM

The API you want to use is the S3 Compatible API, only publicly documented option

dsky•3/18/24, 2:14 PM

so I have to do the AWS signing stuff even though I'm not using S3?

Chaika•3/18/24, 2:14 PM

You will be using the S3 Compatible API, so yes

Chaika•3/18/24, 2:15 PM

There's examples here: https://developers.cloudflare.com/r2/examples/, and lots of libraries which support S3 which mostly just work with R2, as long as you configure them right

Hey guys. Is there any difference in the following two examples? Is one preferred over the other? no

Similar Threads