Cloudflare Developers•14mo ago

What was that again in Cloudflare, where you can restrict access to bucket files URL's without losin

What was that again in Cloudflare, where you can restrict access to bucket files URL's without losing cache features?

101budspencerOP•11/26/24, 4:42 PM

Somehow generating some key and appending it in the URL when accessing it and if the key is not valid it blocks you

1101budspencer What was that again in Cloudflare, where you can restrict access to bucket files...

digitalpoint•11/26/24, 4:45 PM

Are you talking about the is_timed_hmac_valid_v0() function?

https://developers.cloudflare.com/ruleset-engine/rules-language/functions/#hmac-validation

Cloudflare Docs

Functions | Cloudflare Ruleset Engine docs

The Cloudflare Rules language provides functions for manipulating and validating values in an expression:

101budspencerOP•11/26/24, 4:45 PM

Yes

101budspencerOP•11/26/24, 4:45 PM

Thanks!

101budspencerOP•11/26/24, 4:45 PM

https://tenor.com/view/thank-you-baby-snoop-dogg-cameo-thanks-thanks-love-gif-18006774

Tenor

digitalpoint•11/26/24, 4:48 PM

Haven't tried using it with cache, but you would probably need to use the "ignore query string" stuff for the cache level. But also, if you did that, it kind of defeats the purpose of the hmac validation since hitting the cache wouldn't actually be doing the validation.

So not really sure it would actually work (both hmac validation and cache hit).

101budspencerOP•11/26/24, 4:50 PM

Worker runs no matter what I believe, maybe using that instead

digitalpoint•11/26/24, 4:50 PM

Or wait... maybe it would work since I think the Page Rules are before the CDN cache in the traffic sequence

101budspencerOP•11/26/24, 4:50 PM

Or that

digitalpoint•11/26/24, 5:04 PM

That reminds me... I'm using is_timed_hmac_valid_v0()is_timed_hmac_valid_v0() in some places instead of presigned URLs because presigned URLs don't support HTTP/2 or HTTP/3.

If someone is reading things, can we finally get presigned URLs on a protocol that came out more recently than almost 30 years ago.

Ddigitalpoint Haven't tried using it with cache, but you would probably need to use the "ignor...

Chaika•11/26/24, 6:06 PM

fw/waf runs before cache/origin

tezcan•11/26/24, 6:53 PM

hi, I'm trying to upload a file to my r2 bucket with rclone but I get this error. "S3: PutObject, exceeded maximum number of attempts" it tries it twice and uploads successfully on the 3rd attempt but it costs me time. I upload a maximum of 10 new files per day so there's no way I can exceed any limit.

tezcan•11/26/24, 6:58 PM

This is the command I use "rclone sync /remote r2:bucket --progress --transfers=8 --s3-upload-concurrency=8 --s3-upload-cutoff=512M --s3-chunk-size=512M --fast-list --size-only " There was no problem for a week, but today it started giving this error.

tezcan•11/26/24, 6:59 PM

most files +400mb

tezcan•11/26/24, 7:01 PM

Let me try.

tezcan•11/26/24, 7:58 PM

When I looked at the log file in more detail, I saw this "can't copy - source file is being updated". I am connected to the remote server with samba/cifs, I guess it is a problem related to that. The file was actually edited 1 hour ago, but rclone still sees it as being edited. But it doesn't matter. --local-no-check-updated This command solved the problem. Sorry for wasting your time. thanks.

Original message was deleted

bright_parrot_49151•11/27/24, 11:13 AM

Advany•11/27/24, 11:23 AM

I am getting errors for writing and reading to R2 from a workers. I keep getting:

put: We encountered an internal error. Please try again. (10001)

Writing 1 or 2 things per minute orso

Objects are around 200kb orso text

Keys are something like "domain.com/page#target"

am I doing something wrong?

This is really making it difficult to work with R2

Ddave I'm running into a fair amount of these errors: ✘ [ERROR] Error: put: We enco...

Advany•11/27/24, 11:25 AM

where you able to solve this?

Original message was deleted

Advany•11/27/24, 12:00 PM

this one is not doing that. I am thinking of just using the S3 api. Hopefully that is more stable?

Original message was deleted

Advany•11/27/24, 12:11 PM

only doing like 2-3 writes and reads per minute. Different keys.

Original message was deleted

Advany•11/27/24, 12:20 PM

I think so?

AAdvany I think so?

Advany•11/27/24, 12:21 PM

maybe a bit more reads that are also happening in a other worker

Original message was deleted

Advany•11/27/24, 1:02 PM

nope... really dont get the error

AAdvany nope... really dont get the error

Advany•11/27/24, 1:03 PM

strangly... I have the same script, same setup, in a different account. No issues...

AAdvany strangly... I have the same script, same setup, in a different account. No issue...

Advany•11/27/24, 1:08 PM

with higher write/reads

AAdvany where you able to solve this?

dave•11/27/24, 1:11 PM

No. We just retry using aws4fetch if the data really matters. We only use the bindings if it’s a quick script that isn’t important

Ddave No. We just retry using aws4fetch if the data really matters. We only use the bi...

Advany•11/27/24, 3:10 PM

Thanks. Will do the same

Strange enough same worker, more traffic working fine in one account and giving errors in the other one

AAdvany Thanks. Will do the same Strange enough same worker, more traffic working fine ...

dave•11/27/24, 3:33 PM

Yeah, I didn't see much of a pattern either.

para•11/27/24, 5:15 PM

Hi, we're switching from S3 to R2 and have about 300k original images. On top of this, we also have thumbnail versions stored so 600k images in total. Does it make economical sense to store both versions with R2 compared to using transforms from the original image? The thought process is that the monthly storage would cost less than the possible transforms.

maurus•11/27/24, 5:40 PM

Hi, I'm trying to debug CORS on R2. I've configured a CORS policy on my R2 bucket in the Cloudflare dashboard, but when doing a preflight request I'm getting 403

<?xml version="1.0" encoding="UTF-8"?><Error><Code>Unauthorized</Code><Message>CORS not configured for this bucket</Message></Error>

<?xml version="1.0" encoding="UTF-8"?><Error><Code>Unauthorized</Code><Message>CORS not configured for this bucket</Message></Error>

(I'm using presigned URLs)

Does it take some time until CORS policy changes are propagated?

Original message was deleted

para•11/27/24, 6:44 PM

Image size average is 150kb, so should be around ~100 GB.

para•11/27/24, 6:48 PM

Perfect, makes this much easier. Thanks for your insight.

Original message was deleted

Advany•11/27/24, 9:14 PM

You are really awesome with all the help you give people with r2

prstsn•11/27/24, 10:35 PM

Hello, i'm new here.
I have trouble with uploading files in R2 with correct key.
When i try to generate upload presign url (that i'm using in browser to upload file) with getSignedUrl it appends bucket name in the end of the key automatically so it creates folder with key i passed and makes file in it with bucket name. Key looks something like this: avatar.png/bucketname instead of: avatar.png.
If i directly upload with putObject from node, it uploads file with correct key.
So only generating presignUrl has problem.
I have forcePathStyle: true in my s3client config.

Has anyone ever had problem like this?

imag•11/27/24, 10:50 PM

Hey guys a bit of an odd question but is there any way to get an objects size (content length if thats what you'd call it) within a cf worker? I see that the aws-sdk's might let you do it but Im looking towards handling it inside a worker binding specifically, anyone come across this before?

roter_schnee•11/28/24, 1:25 AM

Hey, guys, newbie here.
I am going to publish a site with cf pages.
I do not expect to get a lot of traffic (total monthly visitors under 3-4k), 90% of traffic is expected to come from one region (Eastern Eu).
The site will be serving some static images (no image transformations, 40-60 images total, 2-3mb each).
So I am not sure which approach fits better described case. R2 with custom domain or CF Images? Would appreciate any advice.

Rroter_schnee Hey, guys, newbie here. I am going to publish a site with cf pages. I do not exp...

1984 Ford Laser•11/28/24, 2:48 AM

R2 with custom domain will be cheaper, but a bit more effort

TrueHeads•11/28/24, 3:35 AM

Kind of an emergency here.

I added my domain to a bucket via "public access" and then found out I can just use CORS policies to allow gets to the bucket, so I removed the domain from public access list, and now my website is no longer live and 404s...

Is this temporary???

TTrueHeads Kind of an emergency here. I added my domain to a bucket via "public access" an...

Chaika•11/28/24, 3:44 AM

Adding your domain as a Custom Domain to your r2 bucket takes over the entire hostname, it prompts you if it's replacing a record as well. You'll want to restore the record that was there before. If it was a Pages Custom Domain, just delete and readd it as a Pages Custom Domain. If it was something else/a normal origin, you could check Audit Logs: https://developers.cloudflare.com/fundamentals/setup/account/account-security/review-audit-logs/ if you don't remember the old value

Cloudflare Docs

Review audit logs | Cloudflare Fundamentals docs

Audit logs summarize the history of changes made within your Cloudflare account. Audit logs include account level actions like login, as well as zone configuration changes.

CChaika Adding your domain as a Custom Domain to your r2 bucket takes over the entire ho...

TrueHeads•11/28/24, 3:46 AM

I just added back the cname for example.comexample.com to example.pages.devexample.pages.dev and its back...

PHEW

ItsWendell•11/28/24, 9:32 AM

Hi all,

I'm trying to migrate the files of bucket A into bucket B, but both buckets are marked as EU and don't show up in the Data Migration tool. Is this on purpose? And what other method could I use to migrate these files? I need to keep the same folder structure as we have in bucket A.

prstsn•11/28/24, 11:18 AM

hey guys.
do anyone know why key looks like this in picture: Sandro_7d260905-9da2-4dab-9b02-108af2fc5841.jpeg/mediaSandro_7d260905-9da2-4dab-9b02-108af2fc5841.jpeg/media instead what i set while generating presign url: Sandro_7d260905-9da2-4dab-9b02-108af2fc5841.jpegSandro_7d260905-9da2-4dab-9b02-108af2fc5841.jpeg. My bucket name is media and it automatically appends at the end of the key.

prstsn•11/28/24, 11:22 AM

this is my code. even if i hardcode Key: example.pngexample.png it automatically appends bucket name at the end of the key.

prstsn•11/28/24, 11:23 AM

mediamedia is my bucket name. mime type of that file is image/png or image/jpg

prstsn•11/28/24, 11:31 AM

I thought endpoint was a problem. Because in documentation there is just domain endpoint to set in s3 client config. but R2 gives me that endpoint on buckets and it has /bucket_name at the end like this: https://my_account_id.r2.cloudflarestorage.com/bucket_name

if i remove bucket name at the end of endpoint it doesn't works and throws access denied error and still addes bucket name at the end of the key

so it doesn't matter.

prstsn•11/28/24, 11:32 AM

on client side i just make put request on that presigned url. i don't customize anything so i think it's correct

prstsn•11/28/24, 11:32 AM

i even tried with curl and i got same result

Original message was deleted

prstsn•11/28/24, 11:34 AM

Is that correct to have bucketName at the end of endpoint?
because in R2 documentation examples there is not /bucket_name at the end of endpoint.

What was that again in Cloudflare, where you can restrict access to bucket files URL's without losin

Similar Threads

Similar Threads

Similar Threads