If the download is going through a Worker, you can decompress it on the fly.

Ddave If the download is going through a Worker, you can decompress it on the fly.

D

daveOP•1/27/24, 3:13 PM

Or if it’s being viewed in a browser

Original message was deleted

D

daveOP•1/27/24, 3:20 PM

What’s annoying is that the etag won’t be the md5 hash of the entire file if you do chunked uploads.

AAnay-208 So do I’ve to use a worker for it then ?

D

daveOP•1/27/24, 4:24 PM

your Worker won't be able to accept POST'd 10GB file

AAnay-208 any alternatives

D

daveOP•1/27/24, 4:25 PM

you can sign the

Content-Length

Content-Length

Content-Length

Content-Length header, but this requires you to know the size that the user will upload.

AAnay-208 I need a max file size of 5 MB on presigned url. So instead of presigned url, I ...

D

daveOP•1/27/24, 4:26 PM

correct.

AAnay-208 I need a max file size of 5 MB on presigned url. So instead of presigned url, I ...

D

daveOP•1/27/24, 4:28 PM

you probably also want to use the

If-Unmodified-Since

If-Unmodified-Since

If-Unmodified-Since

If-Unmodified-Since header so you don't allow infinite reuploads of the same object.

https://developers.cloudflare.com/r2/api/s3/presigned-urls/#presigned-url-alternative-with-workers

Ddave you can sign the `Content-Length` header, but this requires you to know the size...

E

Erisa•1/27/24, 4:30 PM

This. The way I would do it would be to have the client send the size of the file in the request that generates the presigned URL, then in your backend code you check that the size is not too large and generate a presigned URL with the appropiate content-length

EErisa This. The way I would do it would be to have the client send the size of the fil...

D

daveOP•1/27/24, 4:31 PM

I think the

If-Unmodified-Since

If-Unmodified-Since

If-Unmodified-Since

If-Unmodified-Since header is important too, in case R2 adds S3 Versioning support and @Anay - ping in replies enables it without thinking about this edge case

Ddave I think the `If-Unmodified-Since` header is important too, in case R2 adds S3 Ve...

E

Erisa•1/27/24, 4:32 PM

Yes

AAnay-208 Ya

D

daveOP•1/27/24, 4:32 PM

if you wanna get real fancy, you can also sign the

user-agent

user-agent

user-agent

user-agent header.

AAnay-208 do I add it to request

D

daveOP•1/27/24, 4:33 PM

const signed = await r2.sign(
  new Request(url, {
    method: "PUT",
  }),
  {
    aws: { signQuery: true },
    headers: {
      "If-Unmodified-Since": "Tue, 28 Sep 2021 16:00:00 GMT",
      "Content-Length" "1337",
      "User-Agent": "I am legit 1.2.3.4"
    },
  }
);

const signed = await r2.sign(
  new Request(url, {
    method: "PUT",
  }),
  {
    aws: { signQuery: true },
    headers: {
      "If-Unmodified-Since": "Tue, 28 Sep 2021 16:00:00 GMT",
      "Content-Length" "1337",
      "User-Agent": "I am legit 1.2.3.4"
    },
  }
);

const signed = await r2.sign(
  new Request(url, {
    method: "PUT",
  }),
  {
    aws: { signQuery: true },
    headers: {
      "If-Unmodified-Since": "Tue, 28 Sep 2021 16:00:00 GMT",
      "Content-Length" "1337",
      "User-Agent": "I am legit 1.2.3.4"
    },
  }
);

const signed = await r2.sign(
  new Request(url, {
    method: "PUT",
  }),
  {
    aws: { signQuery: true },
    headers: {
      "If-Unmodified-Since": "Tue, 28 Sep 2021 16:00:00 GMT",
      "Content-Length" "1337",
      "User-Agent": "I am legit 1.2.3.4"
    },
  }
);

D

daveOP•1/27/24, 4:36 PM

you could/should also limit the content-type header if you know it ahead of time.

AAnay-208 Do I've to use aws4fetch library, or it can't be used s3-request-presigner

D

daveOP•1/27/24, 4:36 PM

oh yeah I'd use aws4fetch

AAnay-208 Do I've to use aws4fetch library, or it can't be used s3-request-presigner

D

daveOP•1/27/24, 4:37 PM

for aws4fetch, you need to set

allHeaders: true

allHeaders: true

allHeaders: true

allHeaders: true so it'll sign all headers, even the "non-standard" ones like

User-Agent

User-Agent

User-Agent

User-Agent

D

daveOP•1/27/24, 4:38 PM

if you wanna see what special headers S3 can use, see:

https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObject.html#API_PutObject_RequestSyntax

PutObject - Amazon Simple Storage Service

Adds an object to a bucket.

D

daveOP•1/27/24, 4:39 PM

you could even sign the

Cookie

Cookie

Cookie

Cookie header

AAnay-208 before returning the presigned uri, the user needs to be authenticated to the ap...

D

daveOP•1/27/24, 4:41 PM

do you usually send cookie or an authorization header on requests?

AAnay-208 on the frontend, it gets the cookie, and sends it in the headers on requests.

D

daveOP•1/27/24, 4:45 PM

ah hmm now that I think about it, don't sign the cookie header, since presigned URLs aren't gonna be on your domain

AAnay-208 Is it a good idea to set this to If-Unmodified-Since ```js const signed = await ...

D

daveOP•1/27/24, 4:54 PM

I think that makes sense.

I would also sign the

Content-Disposition

Content-Disposition

Content-Disposition

Content-Disposition to enforce

Content-Disposition: attachment

Content-Disposition: attachment

Content-Disposition: attachment

Content-Disposition: attachment. And yeah, I'd sign

User-Agent

User-Agent

User-Agent

User-Agent if it's static for your application.

AAnay-208 And if I’m only accepting only image files any headers to sign?

D

daveOP•1/27/24, 4:55 PM

do you know what image type?

D

daveOP•1/27/24, 4:55 PM

if you know the type, then

Content-Type: image/jpeg

Content-Type: image/jpeg

Content-Type: image/jpeg

Content-Type: image/jpeg

D

daveOP•1/27/24, 4:59 PM

const signed = await r2.sign(
      new Request(url, {
        method: "PUT",
      }),
      {
        aws: { signQuery: true },
        headers: {
          "If-Unmodified-Since": new Date().toUTCString(),
          "Content-Length": "1337",
          "Content-Type": "image/jpeg",
          "Content-Disposition": "attachment",
          "Cache-Control": "private, immutable, max-age=31536000",
          "User-Agent": "myapp ver 1337"
        },
      }
    );

const signed = await r2.sign(
      new Request(url, {
        method: "PUT",
      }),
      {
        aws: { signQuery: true },
        headers: {
          "If-Unmodified-Since": new Date().toUTCString(),
          "Content-Length": "1337",
          "Content-Type": "image/jpeg",
          "Content-Disposition": "attachment",
          "Cache-Control": "private, immutable, max-age=31536000",
          "User-Agent": "myapp ver 1337"
        },
      }
    );

const signed = await r2.sign(
      new Request(url, {
        method: "PUT",
      }),
      {
        aws: { signQuery: true },
        headers: {
          "If-Unmodified-Since": new Date().toUTCString(),
          "Content-Length": "1337",
          "Content-Type": "image/jpeg",
          "Content-Disposition": "attachment",
          "Cache-Control": "private, immutable, max-age=31536000",
          "User-Agent": "myapp ver 1337"
        },
      }
    );

const signed = await r2.sign(
      new Request(url, {
        method: "PUT",
      }),
      {
        aws: { signQuery: true },
        headers: {
          "If-Unmodified-Since": new Date().toUTCString(),
          "Content-Length": "1337",
          "Content-Type": "image/jpeg",
          "Content-Disposition": "attachment",
          "Cache-Control": "private, immutable, max-age=31536000",
          "User-Agent": "myapp ver 1337"
        },
      }
    );

The

Content-Disposition

Content-Disposition

Content-Disposition

Content-Disposition and

Cache-Control

Cache-Control

are totally personal preference and depends on the context of what you do with these images. And yeah

User-Agent

User-Agent

User-Agent

User-Agent is optional. cc @Anay - ping in replies

D

daveOP•1/27/24, 5:00 PM

Content-Disposition: attachment

Content-Disposition: attachment

Content-Disposition: attachment

Content-Disposition: attachment will prevent browsers from rendering the image inline

Original message was deleted

Z

Zora•1/27/24, 5:00 PM

Expensive if you want to store a ton of data

AAnay-208 And I also wanted it such that images are automatically deleted after 1 week, un...

H

Hello, I’m Allie!•1/27/24, 5:28 PM

You can set up Lifecycle rules, but they work based on upload time, not last read

H

Hello, I’m Allie!•1/27/24, 5:29 PM

So you would have to build your own system to auto-delete files. I would probably spawn a Durable Object on upload, with an alarm set for a week from now. Whenever the DO receives a request, it resets the alarm. If it reaches 0, it deletes itself and the file

AAnay-208 I've a question, why have you set max-age so high(1 year I think)

D

daveOP•1/27/24, 5:30 PM

If the file names are always unique, there’s no reason to make a client waste their bandwidth downloading the file again if they can cache it for themselves.

If the download is going through a Worker, you can decompress it on the fly.

Similar Threads

Similar Threads

Similar Threads