Does anyone know the expiration time of S3 Presigned URLs? If the client has not completed the uploa

Ssin Does anyone know the expiration time of S3 Presigned URLs? If the client has not...

E

Erisa•11/9/23, 5:07 PM

Start time, ongoing connections are not interrupted and complete like normal.
The same is true for downloads; you can start downloading a file from presigned url and will keep downloading even if it expires halfway

Ccake We are seeing that R2 API tokens / secrets are expiring, and we're getting `aws....

S

Sid•11/9/23, 5:26 PM

That's to be expected no? Your token expired so API access was revoked? I may be misunderstanding

KKentaro [R2 API Token] I received "Access Denied" for S3 API when I use "Object Read & W...

S

Sid•11/9/23, 5:27 PM

What action are you issuing? Object tokens will not allow bucket-level configuration actions

N

nitsky•11/9/23, 5:28 PM

Hi, I have a REST API where a subset of the routes return objects from R2 which can be made publicly available. Rather than reading the objects from my API to serve those requests, I would like to serve them directly from Cloudflare with the public bucket feature. However, I still want all other routes to hit my API servers. Is this possible? Thanks!

Nnitsky Hi, I have a REST API where a subset of the routes return objects from R2 which ...

S

Sid•11/9/23, 5:31 PM

Note that making a bucket public by attaching a custom domain will make the entire bucket public. There currently isn't a way to keep certain prefixes private.
I'd recommend putting up a Worker (or using your existing server) in front of your bucket to decide if you want to serve a certain object

SSid Note that making a bucket public by attaching a custom domain will make the enti...

N

nitsky•11/9/23, 5:32 PM

In my case, everything can be public! I am hoping to take advantage of that to reduce costs.

Nnitsky In my case, everything can be public! I am hoping to take advantage of that to r...

S

Sid•11/9/23, 5:33 PM

Ah, in that case, have you looked into Page Rules on your domain?

Original message was deleted

C

Chaika•11/9/23, 7:37 PM

This is the channel for Cloudflare's R2 Object Storage. If you have a question about AWS billing, you should probably reach out to AWS, not much we can do here.

Sshep what are the rate limits on r2?

W

WakaTime•11/9/23, 9:06 PM

did you ever find an answer? I'm wondering the same about R2 max requests per second

S

shep•11/9/23, 9:06 PM

yeah

S

shep•11/9/23, 9:06 PM

theres no rate limit

S

shep•11/9/23, 9:06 PM

well i didnt hit any

S

shep•11/9/23, 9:06 PM

so was doing 10k batches

W

WakaTime•11/9/23, 9:07 PM

what avg req/sec did you make?

W

WakaTime•11/9/23, 9:07 PM

those were writes?

W

WakaTime•11/9/23, 9:10 PM

1k/sec reads would be enough for me, as long as latency is low. i'll just try it out and see compared to s3

S

shep•11/9/23, 9:10 PM

it was 10k a sec

U

Unsmart•11/9/23, 9:10 PM

There are no rate limits on R2. But you can break the service if you are in the range of like ~5-10k RPS potentially

S

shep•11/9/23, 9:10 PM

well this was just for migration to r2

S

shep•11/9/23, 9:11 PM

but im prob doing like 10k /10sec class b

S

shep•11/9/23, 9:11 PM

the hotlink protection is good also

Sshep the hotlink protection is good also

W

WakaTime•11/9/23, 9:16 PM

oh, these files are only accessed via API not public

S

shep•11/9/23, 9:21 PM

yeah then what the mod said

Sshep it was 10k a sec

Z

Zora•11/9/23, 11:16 PM

That's very nice, for comparison backblaze b2 has a request rate limit of double digits before it starts giving 429 errors

Z

Zora•11/9/23, 11:16 PM

I learned that the hard way

K

kian•11/10/23, 12:04 AM

Same as gzip.

K

kian•11/10/23, 12:05 AM

Cloudflare will decompress it (if necessary) based on the client's Accept-Encoding header

S

snow21•11/10/23, 12:08 AM

Is anyone using R2 storage with Flutter or Dart?

I

iot•11/10/23, 5:20 AM

i uses R2 for storing media files for my production server (django rest API's ), mainly i do converting all types of files to pdf and saving in R2 storage, while converting jpg, png theree is no problem with that , but when i do for MS files (doc, docx, xls, xlsx, ppt, pptx) , it gives me problem.

I think there is issue with output file url ( as it seems path containg X-Amz-Algorithm=, X-Amz-Credential=, X-Amz-Expires=, X-Amz-SignedHeaders=host&X-Amz-Signature=)etc, please give me the solution, what should i do for that?

ZZora That's very nice, for comparison backblaze b2 has a request rate limit of double...

W

WakaTime•11/10/23, 8:50 AM

DigitalOcean Spaces has a limit of 500 req/sec per bucket and 1500 req/sec for all buckets.. so only R2 and S3 can compete on request rates

M

Migmac•11/10/23, 9:25 AM

Question: Is there a way I can generate a presigned url with a custom Content-Disposition header?

Something like this?

const r2 = new AwsClient({
  accessKeyId: env.CF_ACCESS_KEY_ID,
  secretAccessKey: env.CF_ACCESS_KEY_SECRET,
})

const url = new URL(`https://${env.BUCKET_NAME}.${env.CF_ACCOUNT_ID}.r2.cloudflarestorage.com`)

url.pathname = obj.path
url.searchParams.set('X-Amz-Expires', `${60 * 60 * 24 * env.STRUCTURE_EXPIRY}`)

const _signed = await r2.sign(new Request(url, { 
  method: 'GET',
  contentDisposition: `attachment; filename="${obj.name}"`, // <------------------------ Something like this?
}), { aws: { signQuery: true } })
const signed = _signed.url

const r2 = new AwsClient({
  accessKeyId: env.CF_ACCESS_KEY_ID,
  secretAccessKey: env.CF_ACCESS_KEY_SECRET,
})

const url = new URL(`https://${env.BUCKET_NAME}.${env.CF_ACCOUNT_ID}.r2.cloudflarestorage.com`)

url.pathname = obj.path
url.searchParams.set('X-Amz-Expires', `${60 * 60 * 24 * env.STRUCTURE_EXPIRY}`)

const _signed = await r2.sign(new Request(url, { 
  method: 'GET',
  contentDisposition: `attachment; filename="${obj.name}"`, // <------------------------ Something like this?
}), { aws: { signQuery: true } })
const signed = _signed.url

const r2 = new AwsClient({
  accessKeyId: env.CF_ACCESS_KEY_ID,
  secretAccessKey: env.CF_ACCESS_KEY_SECRET,
})

const url = new URL(`https://${env.BUCKET_NAME}.${env.CF_ACCOUNT_ID}.r2.cloudflarestorage.com`)

url.pathname = obj.path
url.searchParams.set('X-Amz-Expires', `${60 * 60 * 24 * env.STRUCTURE_EXPIRY}`)

const _signed = await r2.sign(new Request(url, { 
  method: 'GET',
  contentDisposition: `attachment; filename="${obj.name}"`, // <------------------------ Something like this?
}), { aws: { signQuery: true } })
const signed = _signed.url

const r2 = new AwsClient({
  accessKeyId: env.CF_ACCESS_KEY_ID,
  secretAccessKey: env.CF_ACCESS_KEY_SECRET,
})

const url = new URL(`https://${env.BUCKET_NAME}.${env.CF_ACCOUNT_ID}.r2.cloudflarestorage.com`)

url.pathname = obj.path
url.searchParams.set('X-Amz-Expires', `${60 * 60 * 24 * env.STRUCTURE_EXPIRY}`)

const _signed = await r2.sign(new Request(url, { 
  method: 'GET',
  contentDisposition: `attachment; filename="${obj.name}"`, // <------------------------ Something like this?
}), { aws: { signQuery: true } })
const signed = _signed.url

K

kian•11/10/23, 9:27 AM

Provide it in the headers for RequestInit.

K

kian•11/10/23, 9:28 AM

new Request(url, {
  headers: {
    'content-disposition': `attachment; filename="${obj.name}"`
  }
})

new Request(url, {
  headers: {
    'content-disposition': `attachment; filename="${obj.name}"`
  }
})

new Request(url, {
  headers: {
    'content-disposition': `attachment; filename="${obj.name}"`
  }
})

new Request(url, {
  headers: {
    'content-disposition': `attachment; filename="${obj.name}"`
  }
})

K

kian•11/10/23, 9:29 AM

you'd probably want to add

allHeaders: true

allHeaders: true

allHeaders: true

allHeaders: true into the

aws

aws

aws

aws block as well, next to

signQuery: true

signQuery: true

signQuery: true

signQuery: true

K

kian•11/10/23, 9:29 AM

that only signs it though - the client still has to provide it in the request

M

Migmac•11/10/23, 9:43 AM

Awesome! Thank you @kian

=]

=]

=]

=]

N

NestarZ•11/10/23, 11:56 AM

Hello !
Question: any way to assign a custom domain to a R2 bucket using the wrangler command line tool ?

N

NestarZ•11/10/23, 11:57 AM

Or some script that doesnt involve the web dashboard ?

M

Matt•11/10/23, 12:18 PM

Hi, quick question... if I've followed the steps to create a publicly accessible bucket... and everything looks "good": Public URL Access is Allowed, even threw in a test folder and file to make sure there is something there.. but I can't get in

E

Erisa•11/10/23, 12:22 PM

Are you visiting the direct path to a file? Theres no directory listing function so visiting a directory or the root / wont work

EErisa Are you visiting the direct path to a file? Theres no directory listing function...

M

Matt•11/10/23, 12:32 PM

thats excatly what i was doing

sorry new to this.. so there is no way for me to view the files in a bucket.. just a direct link to the file itself?

E

Erisa•11/10/23, 12:33 PM

Not with R2 itself, you would have to deploy a Worker that can do it (for the extra cost of worker requests) e.g. there is a popular one https://github.com/kotx/render/

E

Erisa•11/10/23, 12:35 PM

Yes

E

Erisa•11/10/23, 12:35 PM

Sure but you would have to visit /index.html or add a transform rule to rewrite to it

E

Erisa•11/10/23, 12:35 PM

And thats not R2 itself anymore either

E

Erisa•11/10/23, 12:40 PM

Yeah

E

Erisa•11/10/23, 12:40 PM

Prebaking adds some extra effort when organising and uploading the files, so just a game of tradeoffs as well, good callout

Does anyone know the expiration time of S3 Presigned URLs? If the client has not completed the uploa

Similar Threads