u cant use cloudflare cdn cache for R2, so no matter what you get charged for GetObject operations,

Bbun u cant use cloudflare cdn cache for R2, so no matter what you get charged for Ge...

K

Karew•3/22/24, 5:06 AM

No, you can use the regular Cloudflare CDN with R2 if you connect a custom domain. If you get a

CF-Cache-Status: HIT

CF-Cache-Status: HIT

CF-Cache-Status: HIT

CF-Cache-Status: HIT, there is no charge on R2 at all, no request is made to the R2 backend if the Cloudflare cache serves it.
You're correct about this behavior only being per-datacenter though, so a quiet datacenter will not have cache hits nearly as often.

M

mickiewicz12•3/22/24, 6:23 AM

Hi,

I have an application for generating images connected with R2.
I noticed that some of the output links with images are delayed up to several minutes, until then, a 404 page is displayed
I have my own domain connected and all rate limits and protections for this bucket are disabled in the WAF tab.
Users generate about 3 images/second. Is it possible that this is too much for R2 or am I missing something?
I also used AWS S3 and Google storage and did not observe a similar experience even under much higher loads.

Thanks in advance.

P

paulrogers_sc•3/22/24, 1:21 PM

Greetings, I'm running into trouble trying to remove DNS for a previously R2-connected-domain, yet the bucket has been deleted.
So trying to click through this Domain Access link just produces "The specified bucket does not exist. (Code: 10006) " and a partially rendered bucket page.
https://dash.cloudflare.com/b5317876d8ea502bb62c977b940cab89/r2/default/buckets/sc-user-content/settings

Recreating the bucket is also blocked. What's the next step to get this resolved?

H

heyjay•3/22/24, 1:37 PM

I am using a CDN that uses R2. I upload to it using the app called Forklift. The CDN asset produces a 404 error unless I add a query param.

curl -I https://cdn.fullqueuedeveloper.com/S5/S5E9-video-1.mp4\?hi

curl -I https://cdn.fullqueuedeveloper.com/S5/S5E9-video-1.mp4\?hi

curl -I https://cdn.fullqueuedeveloper.com/S5/S5E9-video-1.mp4\?hi

curl -I https://cdn.fullqueuedeveloper.com/S5/S5E9-video-1.mp4\?hi will give me a 200 response but

curl -I https://cdn.fullqueuedeveloper.com/S5/S5E9-video-1

curl -I https://cdn.fullqueuedeveloper.com/S5/S5E9-video-1

curl -I https://cdn.fullqueuedeveloper.com/S5/S5E9-video-1

curl -I https://cdn.fullqueuedeveloper.com/S5/S5E9-video-1 will give me a 404 response

It's also only

*.mp4

*.mp4

*.mp4

*.mp4 files this happens with. My

*.png

*.png

*.png

*.png files work great

Hheyjay I am using a CDN that uses R2. I upload to it using the app called Forklift. The...

C

Chaika•3/22/24, 1:40 PM

Your 404s are being cached, and also

curl -I https://cdn.fullqueuedeveloper.com/S5/S5E9-video-1

will never match because it's missing the .mp4 extension entirely, which is part of the object key, should be

https://cdn.fullqueuedeveloper.com/S5/S5E9-video-1.mp4

https://cdn.fullqueuedeveloper.com/S5/S5E9-video-1.mp4

https://cdn.fullqueuedeveloper.com/S5/S5E9-video-1.mp4

https://cdn.fullqueuedeveloper.com/S5/S5E9-video-1.mp4

H

heyjay•3/22/24, 1:40 PM

Yeah sorry... deleted that...

H

heyjay•3/22/24, 1:41 PM

curl -I https://cdn.fullqueuedeveloper.com/S5/S5E9-video-1.mp4

curl -I https://cdn.fullqueuedeveloper.com/S5/S5E9-video-1.mp4

curl -I https://cdn.fullqueuedeveloper.com/S5/S5E9-video-1.mp4

curl -I https://cdn.fullqueuedeveloper.com/S5/S5E9-video-1.mp4

H

heyjay•3/22/24, 1:41 PM

It gives me a 404

H

heyjay•3/22/24, 1:41 PM

❯ curl -I https://cdn.fullqueuedeveloper.com/S5/S5E9-video-1.mp4
HTTP/2 404 
date: Fri, 22 Mar 2024 13:40:53 GMT
content-type: text/html
vary: Accept-Encoding
cache-control: max-age=14400
cf-cache-status: HIT
age: 562992
report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v4?s=VjFNvapiIcOqkZjiRbA%2FM1%2BbpnwVbcLQzgwoFfPSqCo%2Bq8Ex0FFUll2jk5qAMb0SeVp%2FDBlR4EMibQ%2FdODzzVa%2FdNa9UaTA2QV7mep9U0wob9fYUgkU14CUYjyeunZkJ5rtWoN%2FFYDCtO9rYNA%3D%3D"}],"group":"cf-nel","max_age":604800}
nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
server: cloudflare
cf-ray: 8686989d2d358d2c-DFW
alt-svc: h3=":443"; ma=86400

❯ curl -I https://cdn.fullqueuedeveloper.com/S5/S5E9-video-1.mp4
HTTP/2 404 
date: Fri, 22 Mar 2024 13:40:53 GMT
content-type: text/html
vary: Accept-Encoding
cache-control: max-age=14400
cf-cache-status: HIT
age: 562992
report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v4?s=VjFNvapiIcOqkZjiRbA%2FM1%2BbpnwVbcLQzgwoFfPSqCo%2Bq8Ex0FFUll2jk5qAMb0SeVp%2FDBlR4EMibQ%2FdODzzVa%2FdNa9UaTA2QV7mep9U0wob9fYUgkU14CUYjyeunZkJ5rtWoN%2FFYDCtO9rYNA%3D%3D"}],"group":"cf-nel","max_age":604800}
nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
server: cloudflare
cf-ray: 8686989d2d358d2c-DFW
alt-svc: h3=":443"; ma=86400

❯ curl -I https://cdn.fullqueuedeveloper.com/S5/S5E9-video-1.mp4
HTTP/2 404 
date: Fri, 22 Mar 2024 13:40:53 GMT
content-type: text/html
vary: Accept-Encoding
cache-control: max-age=14400
cf-cache-status: HIT
age: 562992
report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v4?s=VjFNvapiIcOqkZjiRbA%2FM1%2BbpnwVbcLQzgwoFfPSqCo%2Bq8Ex0FFUll2jk5qAMb0SeVp%2FDBlR4EMibQ%2FdODzzVa%2FdNa9UaTA2QV7mep9U0wob9fYUgkU14CUYjyeunZkJ5rtWoN%2FFYDCtO9rYNA%3D%3D"}],"group":"cf-nel","max_age":604800}
nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
server: cloudflare
cf-ray: 8686989d2d358d2c-DFW
alt-svc: h3=":443"; ma=86400

❯ curl -I https://cdn.fullqueuedeveloper.com/S5/S5E9-video-1.mp4
HTTP/2 404 
date: Fri, 22 Mar 2024 13:40:53 GMT
content-type: text/html
vary: Accept-Encoding
cache-control: max-age=14400
cf-cache-status: HIT
age: 562992
report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v4?s=VjFNvapiIcOqkZjiRbA%2FM1%2BbpnwVbcLQzgwoFfPSqCo%2Bq8Ex0FFUll2jk5qAMb0SeVp%2FDBlR4EMibQ%2FdODzzVa%2FdNa9UaTA2QV7mep9U0wob9fYUgkU14CUYjyeunZkJ5rtWoN%2FFYDCtO9rYNA%3D%3D"}],"group":"cf-nel","max_age":604800}
nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
server: cloudflare
cf-ray: 8686989d2d358d2c-DFW
alt-svc: h3=":443"; ma=86400

C

Chaika•3/22/24, 1:41 PM

see the cache there?

C

Chaika•3/22/24, 1:42 PM

You're caching 404s. You can use a Cache Rule to override the status by status code. Either to shorter, or no cache at all, or manually call purge whenever you call an upload, ex:

H

heyjay•3/22/24, 1:43 PM

Ohhhh that's good to know!! Thank you so much! This is my first time dealing with R2s and this isn't even mine. I'm working for someone else.

H

heyjay•3/22/24, 1:43 PM

I'll see if he can change the cache rule

C

Chaika•3/22/24, 1:46 PM

Cache Rules are under: Caching -> Cache Rules
https://dash.cloudflare.com/?to=/:account/:zone/caching/cache-rules/new

Easiest way to match it would be

Hostname

Hostname

Hostname

Hostname

equals

equals

equals

equals

cdn.fullqueuedeveloper.com

cdn.fullqueuedeveloper.com

cdn.fullqueuedeveloper.com

cdn.fullqueuedeveloper.com

He's already got an existing Page Rule or Cache Rule for it to cache it that long. If it's a Page Rule, he should recreate it as a Cache Rule. If it's a Cache Rule, he should just modify the existing.

The exact TTL is up to you. Cloudflare caches 404s for 3 minutes by default for example.

M

mickiewicz12•3/22/24, 2:22 PM

Can my problem be also related to 404s caching?

CChaika Cache Rules are under: Caching -> Cache Rules https://dash.cloudflare.com/?to=/:...

H

heyjay•3/22/24, 2:31 PM

Thanks!!

He was able to update the rules and 404s are no longer caching... Now I just need to figure out why content Types are wrong... Uploading a .mp4 file does not give me a content type of

vide/mp4

vide/mp4

vide/mp4

vide/mp4 but maybe that's also a caching error?

Hheyjay Thanks!! He was able to update the rules and 404s are no longer caching... Now ...

I

Isaac McFadyen•3/22/24, 2:35 PM

When you upload a file to R2 you need to specify the content type yourself.

I

Isaac McFadyen•3/22/24, 2:36 PM

It's not auto-detected. I believe it defaults to

binary/octet-stream

binary/octet-stream

binary/octet-stream

binary/octet-stream.

H

heyjay•3/22/24, 2:37 PM

Oh that’s really good to know!

H

heyjay•3/22/24, 2:37 PM

Maybe there’s some settings I need to set in my client to do that for me.

H

heyjay•3/22/24, 2:37 PM

I appreciate the help!

M

maco6517•3/22/24, 2:58 PM

multi-thread copy: failed to write chunk: failed to upload chunk 4 with 5242880 bytes: NoSuchUpload: The specified multipart upload does not exist

M

maco6517•3/22/24, 2:59 PM

I get a rclone error when uploading a file larger than 300mb

Mmaco6517 multi-thread copy: failed to write chunk: failed to upload chunk 4 with 5242880 ...

K

Karew•3/22/24, 7:52 PM

Are you specifically using the R2 config mode on Rclone? Make sure your Rclone is the newest version and set up your connection again, Rclone has a specific choice for R2 connections when setting one up

S

sha386•3/24/24, 1:53 AM

Is it possible to set a key for example im uploading a png file with the key as "textures/blocks/amethyst_cluster.png"

S

sha386•3/24/24, 1:53 AM

https://url/textures/blocks/amethyst_cluster.png

https://url/textures/blocks/amethyst_cluster.png

https://url/textures/blocks/amethyst_cluster.png

https://url/textures/blocks/amethyst_cluster.png

S

sha386•3/24/24, 1:54 AM

however going here gives me a 404

G

grth•3/24/24, 8:10 AM

Hi guys, i run into some troubles when configuring R2 with Vercel image optimization. So i stored my images in r2 and that bucket connects to my domain, which means it refuses to serve from all other domains but mine. But in my nextjs app, i am using Image component from nextjs that makes a request from, i dont know, maybe vercel's server, and it is blocked. I try to configure the CORS of my bucket like allows headers that have user agent, but it still doesnt work. Any ideas?
Thanks

S

slainless•3/24/24, 9:14 AM

hello, does anyone know whats wrong with my r2 connection, i can do HeadBucket just fine, but it returns 403 when putting object (via PutObject)

Sslainless hello, does anyone know whats wrong with my r2 connection, i can do HeadBucket j...

S

slainless•3/24/24, 9:15 AM

permission is correct, i think

Original message was deleted

H

Hello, I’m Allie!•3/24/24, 10:46 AM

Is this what it is supposed to look like?

Original message was deleted

H

Hello, I’m Allie!•3/24/24, 10:52 AM

Might be that you have the old version cached? I do see it:

Original message was deleted

H

Hello, I’m Allie!•3/24/24, 10:55 AM

Do you see a

CF-Cache-Status

CF-Cache-Status

CF-Cache-Status

CF-Cache-Status header in the response?

Original message was deleted

H

Hello, I’m Allie!•3/24/24, 10:57 AM

Can I see the Request Headers too?

OOhyzd Multi region storage will be huge. I'd be happy to pay double for storage to hav...

O

Ohyzd•3/24/24, 11:53 AM

Anyone interested in this should check out https://www.tigrisdata.com
I'm using R2 for the time being, but keeping an eye out on this

Tigris | Globally Distributed S3-Compatible Object Storage

Tigris is a globally distributed S3-compatible object storage service that provides low latency anywhere in the world, enabling developers to store and access any amount of data for a wide range of use cases.

K

kian•3/24/24, 5:45 PM

Use fast-xml-parser or just use aws-sdk-js

E

Erisa•3/24/24, 6:56 PM

aws-sdk-js does work on workers yes

E

Erisa•3/24/24, 6:57 PM

aws4fetch is preferred due to simplicity but in this case it sounds like you need the extra niceties offered by the aws sdk

R

RAGman•3/24/24, 8:22 PM

Hi, Trying to get presigned url to work using s3 compatible api. Generate a PUT presigned url, but it will not work, returning a 403 Forbidden error in Postman, but 400 - Bad Request when trying to upload from browser. I should note that I have presigned urls working with several other services but R2 won't work. My config is: const CF_s3Client = new S3Client({
region: "eu",
endpoint: process.env.CLOUDFLARE_EU_ENDPOINT,
credentials: {
accessKeyId: process.env.CLOUDFLARE_ACCESS_KEY,
secretAccessKey: process.env.CLOUDFLARE_SECRET_ACCESS_KEY,
},
signatureVersion: "v4",
});

M

mickiewicz12•3/24/24, 9:38 PM

Hi, I'm experiencing delays when uploading more than 5 images per second. How can I increase the speed of simultaneous uploads? Should I split it into two buckets?

D

Deliable•3/24/24, 9:47 PM

Hey!

D

Deliable•3/24/24, 9:47 PM

I'm trying to use the S3 client in Java, what should I specify for region?

u cant use cloudflare cdn cache for R2, so no matter what you get charged for GetObject operations,

Similar Threads

Similar Threads

Similar Threads