lol those are just tags for things you're interested in, anyone can get them <id:customize> , althou

CChaika lol those are just tags for things you're interested in, anyone can get them <id...

M

Mehanika•10/20/23, 8:47 PM

Oh, haha! Did not know that. Thank you for clarifying two things at once!
Appreciate it once again!

S

shep•10/21/23, 2:54 AM

hey so when i curl test my upload it works fine

S

shep•10/21/23, 2:54 AM

im thinking maybe my headers are causing issues

S

shep•10/21/23, 2:56 AM

PUT /pictures/ava/-cbcc-4f98-acaa-81c556b59712_227f8f9c-6da4-4430-8f6b-1c56c869e6e8.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=e45bd1f0b9630ed2055abb90de24%2F20231021%2Fauto%2Fs3%2Faws4_request&X-Amz-Date=20231021T025045Z&X-Amz-Expires=300&X-Amz-Signature=7a90fffd87b621c7f5a3fe2d03621beb5275fa8fc4aa3c75235892783aeaff9c&X-Amz-SignedHeaders=host&x-id=PutObject HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
Authorization: Bearer jwtxx
Connection: keep-alive
Content-Length: 186746
Content-Type: image/jpeg
DNT: 1
Host: pictures.a88a0a0ec0a380338bfd981ac867c85a.r2.cloudflarestorage.com
Origin: http://localhost:5174
Referer: http://localhost:5174/
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: cross-site
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36
sec-ch-ua: "Not=A?Brand";v="99", "Chromium";v="118"
sec-ch-ua-mobile: ?0
sec-ch-ua-platform: "macOS"

PUT /pictures/ava/-cbcc-4f98-acaa-81c556b59712_227f8f9c-6da4-4430-8f6b-1c56c869e6e8.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=e45bd1f0b9630ed2055abb90de24%2F20231021%2Fauto%2Fs3%2Faws4_request&X-Amz-Date=20231021T025045Z&X-Amz-Expires=300&X-Amz-Signature=7a90fffd87b621c7f5a3fe2d03621beb5275fa8fc4aa3c75235892783aeaff9c&X-Amz-SignedHeaders=host&x-id=PutObject HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
Authorization: Bearer jwtxx
Connection: keep-alive
Content-Length: 186746
Content-Type: image/jpeg
DNT: 1
Host: pictures.a88a0a0ec0a380338bfd981ac867c85a.r2.cloudflarestorage.com
Origin: http://localhost:5174
Referer: http://localhost:5174/
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: cross-site
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36
sec-ch-ua: "Not=A?Brand";v="99", "Chromium";v="118"
sec-ch-ua-mobile: ?0
sec-ch-ua-platform: "macOS"

PUT /pictures/ava/-cbcc-4f98-acaa-81c556b59712_227f8f9c-6da4-4430-8f6b-1c56c869e6e8.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=e45bd1f0b9630ed2055abb90de24%2F20231021%2Fauto%2Fs3%2Faws4_request&X-Amz-Date=20231021T025045Z&X-Amz-Expires=300&X-Amz-Signature=7a90fffd87b621c7f5a3fe2d03621beb5275fa8fc4aa3c75235892783aeaff9c&X-Amz-SignedHeaders=host&x-id=PutObject HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
Authorization: Bearer jwtxx
Connection: keep-alive
Content-Length: 186746
Content-Type: image/jpeg
DNT: 1
Host: pictures.a88a0a0ec0a380338bfd981ac867c85a.r2.cloudflarestorage.com
Origin: http://localhost:5174
Referer: http://localhost:5174/
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: cross-site
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36
sec-ch-ua: "Not=A?Brand";v="99", "Chromium";v="118"
sec-ch-ua-mobile: ?0
sec-ch-ua-platform: "macOS"

PUT /pictures/ava/-cbcc-4f98-acaa-81c556b59712_227f8f9c-6da4-4430-8f6b-1c56c869e6e8.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=e45bd1f0b9630ed2055abb90de24%2F20231021%2Fauto%2Fs3%2Faws4_request&X-Amz-Date=20231021T025045Z&X-Amz-Expires=300&X-Amz-Signature=7a90fffd87b621c7f5a3fe2d03621beb5275fa8fc4aa3c75235892783aeaff9c&X-Amz-SignedHeaders=host&x-id=PutObject HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
Authorization: Bearer jwtxx
Connection: keep-alive
Content-Length: 186746
Content-Type: image/jpeg
DNT: 1
Host: pictures.a88a0a0ec0a380338bfd981ac867c85a.r2.cloudflarestorage.com
Origin: http://localhost:5174
Referer: http://localhost:5174/
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: cross-site
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36
sec-ch-ua: "Not=A?Brand";v="99", "Chromium";v="118"
sec-ch-ua-mobile: ?0
sec-ch-ua-platform: "macOS"

S

shep•10/21/23, 2:56 AM

is there any issues with this?

S

shep•10/21/23, 2:57 AM

[
  {
    "AllowedOrigins": [
      "http://localhost:5174"
    ],
    "AllowedMethods": [
      "PUT",
      "POST",
      "GET",
      "DELETE",
      "HEAD"
    ],
    "AllowedHeaders": [
      "x-requested-by",
      "Content-Type"
    ],
    "ExposeHeaders": []
  }
]

[
  {
    "AllowedOrigins": [
      "http://localhost:5174"
    ],
    "AllowedMethods": [
      "PUT",
      "POST",
      "GET",
      "DELETE",
      "HEAD"
    ],
    "AllowedHeaders": [
      "x-requested-by",
      "Content-Type"
    ],
    "ExposeHeaders": []
  }
]

[
  {
    "AllowedOrigins": [
      "http://localhost:5174"
    ],
    "AllowedMethods": [
      "PUT",
      "POST",
      "GET",
      "DELETE",
      "HEAD"
    ],
    "AllowedHeaders": [
      "x-requested-by",
      "Content-Type"
    ],
    "ExposeHeaders": []
  }
]

[
  {
    "AllowedOrigins": [
      "http://localhost:5174"
    ],
    "AllowedMethods": [
      "PUT",
      "POST",
      "GET",
      "DELETE",
      "HEAD"
    ],
    "AllowedHeaders": [
      "x-requested-by",
      "Content-Type"
    ],
    "ExposeHeaders": []
  }
]

K

kian•10/21/23, 3:12 AM

Why is there an Authorization header meowthinking

S

shep•10/21/23, 3:23 AM

true ill try removing it

K

kian•10/21/23, 3:23 AM

That’s the only thing that stands out personally

S

shep•10/21/23, 3:30 AM

yoooo

S

shep•10/21/23, 3:30 AM

it worked thanks!!

S

shep•10/21/23, 5:54 AM

what are the rate limits on r2?

S

shep•10/21/23, 5:55 AM

im using rclone to transfer a couple million 50kb files from gcp

⚡

⚡ techb0x 🌟•10/21/23, 4:41 PM

hello all, I am trying to download a exe file from a public bucket R2 from a powershell script, like

Invoke-WebRequest -Uri "https://xxxxxxxx" -OutFile "C:\Windows\Temp\xxxxxxx.exe"

Invoke-WebRequest -Uri "https://xxxxxxxx" -OutFile "C:\Windows\Temp\xxxxxxx.exe"

Invoke-WebRequest -Uri "https://xxxxxxxx" -OutFile "C:\Windows\Temp\xxxxxxx.exe"

Invoke-WebRequest -Uri "https://xxxxxxxx" -OutFile "C:\Windows\Temp\xxxxxxx.exe"

, but I get a "forbidden access", from a browser, it works, is there any CORS policies I need to setup ? can you give some advice ? thanks,

R

Raymond•10/21/23, 4:53 PM

I love sippy.

Sshep what are the rate limits on r2?

K

Karew•10/21/23, 7:39 PM

AFAIK there are no rate limits for uploading objects, at least none that rclone will run into

KKarew AFAIK there are no rate limits for uploading objects, at least none that rclone ...

H

Hello, I’m Allie!•10/21/23, 7:40 PM

Suffice to say I saturated my home fiber connection before

rclone

rclone

or

r2

r2

r2

r2 gave up

I

isaacryu•10/21/23, 10:28 PM

I'm running into the following problem trying to copy to an

r2

r2

r2

r2 bucket with

rclone

rclone

2023/10/21 17:52:57 ERROR : Attempt 1/3 failed with 1 errors and: InvalidParameter: 1 validation error(s) found.
- minimum field size of 1, HeadObjectInput.Key.
2023/10/21 17:52:57 ERROR : Attempt 2/3 failed with 1 errors and: InvalidParameter: 1 validation error(s) found.
- minimum field size of 1, HeadObjectInput.Key.
2023/10/21 17:52:57 ERROR : Attempt 3/3 failed with 1 errors and: InvalidParameter: 1 validation error(s) found.
- minimum field size of 1, HeadObjectInput.Key.
2023/10/21 17:52:57 Failed to copyto: InvalidParameter: 1 validation error(s) found.
- minimum field size of 1, HeadObjectInput.Key.

2023/10/21 17:52:57 ERROR : Attempt 1/3 failed with 1 errors and: InvalidParameter: 1 validation error(s) found.
- minimum field size of 1, HeadObjectInput.Key.
2023/10/21 17:52:57 ERROR : Attempt 2/3 failed with 1 errors and: InvalidParameter: 1 validation error(s) found.
- minimum field size of 1, HeadObjectInput.Key.
2023/10/21 17:52:57 ERROR : Attempt 3/3 failed with 1 errors and: InvalidParameter: 1 validation error(s) found.
- minimum field size of 1, HeadObjectInput.Key.
2023/10/21 17:52:57 Failed to copyto: InvalidParameter: 1 validation error(s) found.
- minimum field size of 1, HeadObjectInput.Key.

2023/10/21 17:52:57 ERROR : Attempt 1/3 failed with 1 errors and: InvalidParameter: 1 validation error(s) found.
- minimum field size of 1, HeadObjectInput.Key.
2023/10/21 17:52:57 ERROR : Attempt 2/3 failed with 1 errors and: InvalidParameter: 1 validation error(s) found.
- minimum field size of 1, HeadObjectInput.Key.
2023/10/21 17:52:57 ERROR : Attempt 3/3 failed with 1 errors and: InvalidParameter: 1 validation error(s) found.
- minimum field size of 1, HeadObjectInput.Key.
2023/10/21 17:52:57 Failed to copyto: InvalidParameter: 1 validation error(s) found.
- minimum field size of 1, HeadObjectInput.Key.

2023/10/21 17:52:57 ERROR : Attempt 1/3 failed with 1 errors and: InvalidParameter: 1 validation error(s) found.
- minimum field size of 1, HeadObjectInput.Key.
2023/10/21 17:52:57 ERROR : Attempt 2/3 failed with 1 errors and: InvalidParameter: 1 validation error(s) found.
- minimum field size of 1, HeadObjectInput.Key.
2023/10/21 17:52:57 ERROR : Attempt 3/3 failed with 1 errors and: InvalidParameter: 1 validation error(s) found.
- minimum field size of 1, HeadObjectInput.Key.
2023/10/21 17:52:57 Failed to copyto: InvalidParameter: 1 validation error(s) found.
- minimum field size of 1, HeadObjectInput.Key.

I

isaacryu•10/21/23, 10:28 PM

my config file:

[r2]
type = s3
provider = Cloudflare
access_key_id = ***
secret_access_key = ***
region = auto
endpoint = https://***.r2.cloudflarestorage.com/bucket-name

[r2]
type = s3
provider = Cloudflare
access_key_id = ***
secret_access_key = ***
region = auto
endpoint = https://***.r2.cloudflarestorage.com/bucket-name

[r2]
type = s3
provider = Cloudflare
access_key_id = ***
secret_access_key = ***
region = auto
endpoint = https://***.r2.cloudflarestorage.com/bucket-name

[r2]
type = s3
provider = Cloudflare
access_key_id = ***
secret_access_key = ***
region = auto
endpoint = https://***.r2.cloudflarestorage.com/bucket-name

I

isaacryu•10/21/23, 10:29 PM

rclone copy File.zip r2:/

rclone copy File.zip r2:/

rclone copy File.zip r2:/

rclone copy File.zip r2:/

I

isaacryu•10/21/23, 10:29 PM

I tried

copyto

copyto

copyto

copyto as well but that didn't work

I

isaacryu•10/21/23, 10:49 PM

i figured it out

I

isaacryu•10/21/23, 10:50 PM

endpoint needs to not have bucket name, remade my key with more permissiveness

I

isaacryu•10/21/23, 10:50 PM

that worked

KKarew AFAIK there are no rate limits for uploading objects, at least none that rclone ...

S

shep•10/22/23, 1:29 AM

yeah there wasnt got 300gb transfered superfast

S

SoulHarsh007•10/22/23, 7:37 PM

it looks like accessing R2 with @aws-sdk/client-s3 now requires using:

forcePathStyle: true

forcePathStyle: true

forcePathStyle: true

forcePathStyle: true or else when using PUTObjectCommand, it yeilds:

SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your secret access key and signing method.

SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your secret access key and signing method.

SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your secret access key and signing method.

SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your secret access key and signing method.

The documentation example should be updated to reflect it, if this is actually the case. I can confirm using the forcePathStyle set to Force fixes the issue for me.

K

kian•10/22/23, 7:57 PM

It shouldn't have any difference for R2.

T

Tobias•10/22/23, 8:28 PM

Does R2 actually replicate the data globally? When creating a new bucket, I am prompted to select the region for the bucket (or it autoselects based on my location). The workload I have however requires the data to be available everywhere around the world with low latency

TTobias Does R2 actually replicate the data globally? When creating a new bucket, I am p...

U

Unsmart•10/22/23, 8:30 PM

Data is single region currently but you can use cache to enhance warm reads.

T

Tobias•10/22/23, 8:31 PM

Hmm okay

C

Chris•10/22/23, 8:47 PM

I have an issue with CORS in one of my r2 buckets. I currently have two r2 buckets: one for dev, one for prod. I set each up bucket with similar CORS policies so I can PUT object using presigned URL's into my bucket.

The one for dev looks like this and works. I can PUT from localhost or from

https://dev.name-of-my-app.com

https://dev.name-of-my-app.com

https://dev.name-of-my-app.com

https://dev.name-of-my-app.com just fine.

[
  {
    "AllowedOrigins": [
      "http://localhost:3000",
      "https://dev.name-of-my-app.com"
    ],
    "AllowedMethods": [
      "PUT",
      "GET"
    ],
    "AllowedHeaders": [
      "*"
    ]
  }
]

[
  {
    "AllowedOrigins": [
      "http://localhost:3000",
      "https://dev.name-of-my-app.com"
    ],
    "AllowedMethods": [
      "PUT",
      "GET"
    ],
    "AllowedHeaders": [
      "*"
    ]
  }
]

[
  {
    "AllowedOrigins": [
      "http://localhost:3000",
      "https://dev.name-of-my-app.com"
    ],
    "AllowedMethods": [
      "PUT",
      "GET"
    ],
    "AllowedHeaders": [
      "*"
    ]
  }
]

[
  {
    "AllowedOrigins": [
      "http://localhost:3000",
      "https://dev.name-of-my-app.com"
    ],
    "AllowedMethods": [
      "PUT",
      "GET"
    ],
    "AllowedHeaders": [
      "*"
    ]
  }
]

C

Chris•10/22/23, 8:47 PM

..but I have almost the exact same policy for my prod bucket...

[
  {
    "AllowedOrigins": [
      "http://localhost:3000",
      "https://app.name-of-my-app.com"
    ],
    "AllowedMethods": [
      "PUT",
      "GET"
    ],
    "AllowedHeaders": [
      "*"
    ]
  }
]

[
  {
    "AllowedOrigins": [
      "http://localhost:3000",
      "https://app.name-of-my-app.com"
    ],
    "AllowedMethods": [
      "PUT",
      "GET"
    ],
    "AllowedHeaders": [
      "*"
    ]
  }
]

[
  {
    "AllowedOrigins": [
      "http://localhost:3000",
      "https://app.name-of-my-app.com"
    ],
    "AllowedMethods": [
      "PUT",
      "GET"
    ],
    "AllowedHeaders": [
      "*"
    ]
  }
]

[
  {
    "AllowedOrigins": [
      "http://localhost:3000",
      "https://app.name-of-my-app.com"
    ],
    "AllowedMethods": [
      "PUT",
      "GET"
    ],
    "AllowedHeaders": [
      "*"
    ]
  }
]

...and I can PUT to a presigned URL from

localhost

localhost

localhost

localhost to the prod bucket just fine. BUT, when I try to PUT from

https://app.name-of-my-app.com

https://app.name-of-my-app.com

https://app.name-of-my-app.com

https://app.name-of-my-app.com, I get a 403 CORS error like this...

Access to XMLHttpRequest at 'https://...r2.cloudflarestorage.com/path/to/uploaded/file.txt?X-Amz-Algorithm=...' from origin 'https://app.name-of-my-app.com' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.

Access to XMLHttpRequest at 'https://...r2.cloudflarestorage.com/path/to/uploaded/file.txt?X-Amz-Algorithm=...' from origin 'https://app.name-of-my-app.com' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.

Access to XMLHttpRequest at 'https://...r2.cloudflarestorage.com/path/to/uploaded/file.txt?X-Amz-Algorithm=...' from origin 'https://app.name-of-my-app.com' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.

Access to XMLHttpRequest at 'https://...r2.cloudflarestorage.com/path/to/uploaded/file.txt?X-Amz-Algorithm=...' from origin 'https://app.name-of-my-app.com' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.

C

Chris•10/22/23, 8:48 PM

Does anyone have any idea what is going on? I don't understand why localhost works fine for prod, but

https://app.name-of-my-app.com

https://app.name-of-my-app.com

https://app.name-of-my-app.com

https://app.name-of-my-app.com does not even though it's in the

AllowedOrigins

AllowedOrigins

AllowedOrigins

AllowedOrigins array. Appreciate any help y'all have.

K

kian•10/22/23, 8:48 PM

Look at the response body

C

Chris•10/22/23, 8:50 PM

Okay. Can you let me know what I'm looking for?

C

Chris•10/22/23, 8:51 PM

Appreciate the help.

K

kian•10/22/23, 8:52 PM

Just see if it has an error - i.e if it's

SignatureDoesNotMatch

SignatureDoesNotMatch

SignatureDoesNotMatch

SignatureDoesNotMatch then it isn't a failure due to CORS, it would have failed anyways.

C

Chris•10/22/23, 8:53 PM

I think the response body is empty? There are response headers, but if I try to copy the response body from the JS console, it's just empty.

K

kian•10/22/23, 8:54 PM

I wouldn't expect to see it in the browser because of CORS - if you copy it to cURL and run it there, you'll probably see a body.

C

Chris•10/22/23, 8:57 PM

<Error>
<Code>SignatureDoesNotMatch</Code>
<Message>The request signature we calculated does not match the signature you provided. Check your secret access key and signing method. </Message>
</Error>

<Error>
<Code>SignatureDoesNotMatch</Code>
<Message>The request signature we calculated does not match the signature you provided. Check your secret access key and signing method. </Message>
</Error>

<Error>
<Code>SignatureDoesNotMatch</Code>
<Message>The request signature we calculated does not match the signature you provided. Check your secret access key and signing method. </Message>
</Error>

<Error>
<Code>SignatureDoesNotMatch</Code>
<Message>The request signature we calculated does not match the signature you provided. Check your secret access key and signing method. </Message>
</Error>

C

Chris•10/22/23, 8:57 PM

....so that's something.

K

kian•10/22/23, 8:58 PM

Yeah - so the issue isn't with CORS, that request would have always failed.

C

Chris•10/22/23, 8:58 PM

That's good to know.

C

Chris•10/22/23, 8:58 PM

I'm not understanding why the bucket is cool with localhost but not cool with the app sending it from the browser.

C

Chris•10/22/23, 9:09 PM

...like, I could use a suggestion on how to troubleshoot this. Currently, from localhost, I point to the production app's server. The app server gives me a presigned PUT url. I put from localhost. It works. I go to

app.name-of-my-app.com

app.name-of-my-app.com

app.name-of-my-app.com

app.name-of-my-app.com , which is pointing to the same server as localhost, and when I PUT to the presigned URL, there is a 403.

C

Chris•10/22/23, 9:15 PM

...actually, it looks like I'm proxying to the server from localhost. Let me see if I can connect directly and see if that makes it break locally.

C

Chris•10/22/23, 9:33 PM

Okay. I was able to duplicate the error on localhost. Checked the server settings and didn't see any issues. Went to the Cloudflare R2 dashboard and rolled the API credentials for the prod bucket. Update them in the server. Everything works fine meow. Thanks for your help @kian!

E

EyalAmir•10/23/23, 12:59 AM

I have a question regarding

rclone

rclone

. What's the time unit of

rclone link --expire?

rclone link --expire?

Is it seconds?

EEyalAmir I have a question regarding `rclone`. What's the time unit of `rclone link --exp...

E

Erisa•10/23/23, 1:45 AM

Its in seconds by default. You can get days instead by putting

at the end, for example

--expire 1d

--expire 1d

becomes 86400 seconds (1 day) whereas

--expire 1

--expire 1

is 1 second

F

Flaviu•10/23/23, 5:31 PM

Hi, can somebody help me to configure cloudflare with nodejs multer to upload photo/video?

lol those are just tags for things you're interested in, anyone can get them <id:customize> , althou

Similar Threads