Hi, I have a REST API where a subset of the routes return objects from R2 which can be made publicly

Nnitsky Hi, I have a REST API where a subset of the routes return objects from R2 which ...

S

Sid•11/9/23, 5:31 PM

Note that making a bucket public by attaching a custom domain will make the entire bucket public. There currently isn't a way to keep certain prefixes private.
I'd recommend putting up a Worker (or using your existing server) in front of your bucket to decide if you want to serve a certain object

SSid Note that making a bucket public by attaching a custom domain will make the enti...

N

nitskyOP•11/9/23, 5:32 PM

In my case, everything can be public! I am hoping to take advantage of that to reduce costs.

Nnitsky In my case, everything can be public! I am hoping to take advantage of that to r...

S

Sid•11/9/23, 5:33 PM

Ah, in that case, have you looked into Page Rules on your domain?

Original message was deleted

C

Chaika•11/9/23, 7:37 PM

This is the channel for Cloudflare's R2 Object Storage. If you have a question about AWS billing, you should probably reach out to AWS, not much we can do here.

Sshep what are the rate limits on r2?

W

WakaTime•11/9/23, 9:06 PM

did you ever find an answer? I'm wondering the same about R2 max requests per second

S

shep•11/9/23, 9:06 PM

yeah

S

shep•11/9/23, 9:06 PM

theres no rate limit

S

shep•11/9/23, 9:06 PM

well i didnt hit any

S

shep•11/9/23, 9:06 PM

so was doing 10k batches

W

WakaTime•11/9/23, 9:07 PM

what avg req/sec did you make?

W

WakaTime•11/9/23, 9:07 PM

those were writes?

W

WakaTime•11/9/23, 9:10 PM

1k/sec reads would be enough for me, as long as latency is low. i'll just try it out and see compared to s3

S

shep•11/9/23, 9:10 PM

it was 10k a sec

U

Unsmart•11/9/23, 9:10 PM

There are no rate limits on R2. But you can break the service if you are in the range of like ~5-10k RPS potentially

S

shep•11/9/23, 9:10 PM

well this was just for migration to r2

S

shep•11/9/23, 9:11 PM

but im prob doing like 10k /10sec class b

S

shep•11/9/23, 9:11 PM

the hotlink protection is good also

Sshep the hotlink protection is good also

W

WakaTime•11/9/23, 9:16 PM

oh, these files are only accessed via API not public

S

shep•11/9/23, 9:21 PM

yeah then what the mod said

Sshep it was 10k a sec

Z

Zora•11/9/23, 11:16 PM

That's very nice, for comparison backblaze b2 has a request rate limit of double digits before it starts giving 429 errors

Z

Zora•11/9/23, 11:16 PM

I learned that the hard way

K

kian•11/10/23, 12:04 AM

Same as gzip.

K

kian•11/10/23, 12:05 AM

Cloudflare will decompress it (if necessary) based on the client's Accept-Encoding header

S

snow21•11/10/23, 12:08 AM

Is anyone using R2 storage with Flutter or Dart?

I

iot•11/10/23, 5:20 AM

i uses R2 for storing media files for my production server (django rest API's ), mainly i do converting all types of files to pdf and saving in R2 storage, while converting jpg, png theree is no problem with that , but when i do for MS files (doc, docx, xls, xlsx, ppt, pptx) , it gives me problem.

I think there is issue with output file url ( as it seems path containg X-Amz-Algorithm=, X-Amz-Credential=, X-Amz-Expires=, X-Amz-SignedHeaders=host&X-Amz-Signature=)etc, please give me the solution, what should i do for that?

ZZora That's very nice, for comparison backblaze b2 has a request rate limit of double...

W

WakaTime•11/10/23, 8:50 AM

DigitalOcean Spaces has a limit of 500 req/sec per bucket and 1500 req/sec for all buckets.. so only R2 and S3 can compete on request rates

M

Migmac•11/10/23, 9:25 AM

Question: Is there a way I can generate a presigned url with a custom Content-Disposition header?

Something like this?

const r2 = new AwsClient({
  accessKeyId: env.CF_ACCESS_KEY_ID,
  secretAccessKey: env.CF_ACCESS_KEY_SECRET,
})

const url = new URL(`https://${env.BUCKET_NAME}.${env.CF_ACCOUNT_ID}.r2.cloudflarestorage.com`)

url.pathname = obj.path
url.searchParams.set('X-Amz-Expires', `${60 * 60 * 24 * env.STRUCTURE_EXPIRY}`)

const _signed = await r2.sign(new Request(url, { 
  method: 'GET',
  contentDisposition: `attachment; filename="${obj.name}"`, // <------------------------ Something like this?
}), { aws: { signQuery: true } })
const signed = _signed.url

const r2 = new AwsClient({
  accessKeyId: env.CF_ACCESS_KEY_ID,
  secretAccessKey: env.CF_ACCESS_KEY_SECRET,
})

const url = new URL(`https://${env.BUCKET_NAME}.${env.CF_ACCOUNT_ID}.r2.cloudflarestorage.com`)

url.pathname = obj.path
url.searchParams.set('X-Amz-Expires', `${60 * 60 * 24 * env.STRUCTURE_EXPIRY}`)

const _signed = await r2.sign(new Request(url, { 
  method: 'GET',
  contentDisposition: `attachment; filename="${obj.name}"`, // <------------------------ Something like this?
}), { aws: { signQuery: true } })
const signed = _signed.url

const r2 = new AwsClient({
  accessKeyId: env.CF_ACCESS_KEY_ID,
  secretAccessKey: env.CF_ACCESS_KEY_SECRET,
})

const url = new URL(`https://${env.BUCKET_NAME}.${env.CF_ACCOUNT_ID}.r2.cloudflarestorage.com`)

url.pathname = obj.path
url.searchParams.set('X-Amz-Expires', `${60 * 60 * 24 * env.STRUCTURE_EXPIRY}`)

const _signed = await r2.sign(new Request(url, { 
  method: 'GET',
  contentDisposition: `attachment; filename="${obj.name}"`, // <------------------------ Something like this?
}), { aws: { signQuery: true } })
const signed = _signed.url

const r2 = new AwsClient({
  accessKeyId: env.CF_ACCESS_KEY_ID,
  secretAccessKey: env.CF_ACCESS_KEY_SECRET,
})

const url = new URL(`https://${env.BUCKET_NAME}.${env.CF_ACCOUNT_ID}.r2.cloudflarestorage.com`)

url.pathname = obj.path
url.searchParams.set('X-Amz-Expires', `${60 * 60 * 24 * env.STRUCTURE_EXPIRY}`)

const _signed = await r2.sign(new Request(url, { 
  method: 'GET',
  contentDisposition: `attachment; filename="${obj.name}"`, // <------------------------ Something like this?
}), { aws: { signQuery: true } })
const signed = _signed.url

K

kian•11/10/23, 9:27 AM

Provide it in the headers for RequestInit.

K

kian•11/10/23, 9:28 AM

new Request(url, {
  headers: {
    'content-disposition': `attachment; filename="${obj.name}"`
  }
})

new Request(url, {
  headers: {
    'content-disposition': `attachment; filename="${obj.name}"`
  }
})

new Request(url, {
  headers: {
    'content-disposition': `attachment; filename="${obj.name}"`
  }
})

new Request(url, {
  headers: {
    'content-disposition': `attachment; filename="${obj.name}"`
  }
})

K

kian•11/10/23, 9:29 AM

you'd probably want to add

allHeaders: true

allHeaders: true

allHeaders: true

allHeaders: true into the

aws

aws

aws

aws block as well, next to

signQuery: true

signQuery: true

signQuery: true

signQuery: true

K

kian•11/10/23, 9:29 AM

that only signs it though - the client still has to provide it in the request

M

Migmac•11/10/23, 9:43 AM

Awesome! Thank you @kian

=]

=]

=]

=]

N

NestarZ•11/10/23, 11:56 AM

Hello !
Question: any way to assign a custom domain to a R2 bucket using the wrangler command line tool ?

N

NestarZ•11/10/23, 11:57 AM

Or some script that doesnt involve the web dashboard ?

M

Matt•11/10/23, 12:18 PM

Hi, quick question... if I've followed the steps to create a publicly accessible bucket... and everything looks "good": Public URL Access is Allowed, even threw in a test folder and file to make sure there is something there.. but I can't get in

E

Erisa•11/10/23, 12:22 PM

Are you visiting the direct path to a file? Theres no directory listing function so visiting a directory or the root / wont work

EErisa Are you visiting the direct path to a file? Theres no directory listing function...

M

Matt•11/10/23, 12:32 PM

thats excatly what i was doing

sorry new to this.. so there is no way for me to view the files in a bucket.. just a direct link to the file itself?

E

Erisa•11/10/23, 12:33 PM

Not with R2 itself, you would have to deploy a Worker that can do it (for the extra cost of worker requests) e.g. there is a popular one https://github.com/kotx/render/

E

Erisa•11/10/23, 12:35 PM

Yes

E

Erisa•11/10/23, 12:35 PM

Sure but you would have to visit /index.html or add a transform rule to rewrite to it

E

Erisa•11/10/23, 12:35 PM

And thats not R2 itself anymore either

E

Erisa•11/10/23, 12:40 PM

Yeah

E

Erisa•11/10/23, 12:40 PM

Prebaking adds some extra effort when organising and uploading the files, so just a game of tradeoffs as well, good callout

E

Erisa•11/10/23, 12:41 PM

Yes, the less simple part is creating and uploading index.html that match all your directories and keeping them updated

E

Erisa•11/10/23, 12:56 PM

Great

E

Erisa•11/10/23, 12:56 PM

feel free to publish it when youre comfortable doing that, would be a good resource

A

AydinTheFirst•11/10/23, 1:34 PM

hi im using cloudflare r2 with nodejs im uploading files to server and then uploading them to r2 but whenever i upload a file it gets uploaded with mimetype of application/octet-stream the files are correct on my folder

const g_pdf = new PutObjectCommand({
      Bucket: "fristroop",
      Body: pdf.path,
      Key: `halodergisi/dergiler/${pdf.filename}`,
    });
    await storage.send(g_pdf);

const g_pdf = new PutObjectCommand({
      Bucket: "fristroop",
      Body: pdf.path,
      Key: `halodergisi/dergiler/${pdf.filename}`,
    });
    await storage.send(g_pdf);

const g_pdf = new PutObjectCommand({
      Bucket: "fristroop",
      Body: pdf.path,
      Key: `halodergisi/dergiler/${pdf.filename}`,
    });
    await storage.send(g_pdf);

const g_pdf = new PutObjectCommand({
      Bucket: "fristroop",
      Body: pdf.path,
      Key: `halodergisi/dergiler/${pdf.filename}`,
    });
    await storage.send(g_pdf);

Hi, I have a REST API where a subset of the routes return objects from R2 which can be made publicly

Similar Threads

Similar Threads

Similar Threads