I think it's way more complex then that. A lot of companies use S3 presigned for uploads on websites

I think it's way more complex then that. A lot of companies use S3 presigned for uploads on websites, it really depends on your scope (is this just a generic upload website), or is this an internal company thing so if an admin uploads an img it's whatever, or is this an image all users would see?

CChaika I think it's way more complex then that. A lot of companies use S3 presigned for...

Barry_Based_Benson•8/22/24, 4:46 PM

Exactly

Barry_Based_Benson•8/22/24, 4:46 PM

Depends on who it affects

ChaikaOP•8/22/24, 4:47 PM

Images are small enough that you may want to just proxy them through your backend and check, for larger things makes way less sense for it all go through your backend

digitalpoint•8/22/24, 4:48 PM

I guess it just depends on the content and the site can make that decision. For me, I don’t even trust admins to do things right. Like images they upload have EXIF data stripped and converted to WEBP when it’s uploaded.

digitalpoint•8/22/24, 4:48 PM

And as part of that process, it’s easy enough to check if the input is a valid image since it’s being mucked with anyway.

digitalpoint•8/22/24, 4:50 PM

Same with user uploaded .zip files… if the files are supposed to be in a specific hierarchy in the archive, check that. Obviously you can’t check everything, but a lot you can.

Ddigitalpoint Same with user uploaded .zip files… if the files are supposed to be in a specifi...

ChaikaOP•8/22/24, 4:52 PM

yea, sometimes can do it client side too. I have something where users pick images, the client-side code checks image, and then they can pick to upload directly to bucket. It's b2b software so I really doubt someone would try to jump the check but even if they do it's on them, only they can see their own messed up imgs lol, just want to have sane max upload limits so they can't say "this 1gb file is totally an image"

Barry_Based_Benson•8/22/24, 4:54 PM

And presigned urls make size validation easy

ChaikaOP•8/22/24, 4:56 PM

well you clamp that down to reasonable limits lol

ChaikaOP•8/22/24, 4:57 PM

client says it wants to upload image, requested size 500 mb but images can be up to 25 mb, not going to return a presigned url

ChaikaOP•8/22/24, 4:59 PM

then you have rate limits on the presign endpoint + a short expiration on the presigned sig

ChaikaOP•8/22/24, 4:59 PM

whatever makes sense for your app? This isn't something that proxying it through your backend would magically solve

ChaikaOP•8/22/24, 5:00 PM

ok..?

Barry_Based_Benson•8/22/24, 5:00 PM

And that's a good thing

Barry_Based_Benson•8/22/24, 5:00 PM

Don't need to set a large expiry in case a user wants to upload a large file.

Original message was deleted

ChaikaOP•8/22/24, 5:01 PM

no because you'd have to start a new upload

ChaikaOP•8/22/24, 5:01 PM

you can take as long as you want to upload sure, and that's fine because you're consuming r2's resources/keeping a conn open to them

ChaikaOP•8/22/24, 5:01 PM

but once that upload is done/failed/suceeded, you try again, new sig expiration check

ChaikaOP•8/22/24, 5:02 PM

idk if r2/s3 stuff would let you do that

ChaikaOP•8/22/24, 5:02 PM

would have to look into it, I assume there should be some same max concurrent limit

ChaikaOP•8/22/24, 5:03 PM

yea idk about that, would assume there's some protection there with how many people use it w/ s3 and such but would have to look into it further. At some point w/ Cf ddos prot would kick in

ChaikaOP•8/22/24, 5:05 PM

everything behind cf cdn has automatic protection

Barry_Based_Benson•8/22/24, 5:07 PM

A few second flood of class A ops that CF can mitigate VS processing every single file on your backend.

Like I said you sacrifice some things and win others. It's a pros cons situation

Barry_Based_Benson•8/22/24, 5:07 PM

I'd rather go the presigned url route in this case

ChaikaOP•8/22/24, 5:11 PM

You could sign If-None-Match/If-Unmodified-Since, not sure if that would save you the Class A though. Would have to test if you can do multiple presigned uploads at once

Original message was deleted

ChaikaOP•8/22/24, 5:11 PM

do you know if it lets you actually do more then one at once?

Original message was deleted

ChaikaOP•8/22/24, 5:11 PM

bandwidth is expensive to be proxying it all through another server

ChaikaOP•8/22/24, 5:12 PM

not something I'd want a business to depend on

ChaikaOP•8/22/24, 5:12 PM

lots of extra work for lb/reliability/etc

Barry_Based_Benson•8/22/24, 5:13 PM

You seem very security conscious which is good but it should be balanced with you resources

ChaikaOP•8/22/24, 5:13 PM

lb is a decent extra cost plus maintaining/paying for all the machines

Barry_Based_Benson•8/22/24, 5:14 PM

Do you still use a lock on the door to your house? What if someone picks that? You gonna use a million different doors and locks and security systems too?

ChaikaOP•8/22/24, 5:14 PM

I think there's security conscious and security paranoid, maybe people use presigned a bit too much but there's def lots of envs you can use them in without too much worry

ChaikaOP•8/22/24, 5:15 PM

except doing all the work to setup a server to proxy all the uploads/check them/lb/etc is a lot of dev work vs just making a worker returning a presigned url

Original message was deleted

dave•8/22/24, 5:15 PM

you need more dev costs to support more infra

Barry_Based_Benson•8/22/24, 5:17 PM

The more your have, logically the harder it can be to maintain

Barry_Based_Benson•8/22/24, 5:17 PM

Maybe for now they're flat

ChaikaOP•8/22/24, 5:18 PM

I'll try later today. It has to go to the bucket/DO so I would assume it'd reject but not 100%

Barry_Based_Benson•8/22/24, 5:19 PM

Doesn't multiple putobject requests usually cause an overwrite if the object is already there anyways? Regardless of concurrency

Barry_Based_Benson•8/22/24, 5:24 PM

If its not already a feature, it would nice to put ip in the header for generating the url

Barry_Based_Benson•8/22/24, 5:25 PM

Which can limit an attack

ChaikaOP•8/22/24, 5:27 PM

S3 API is IPv6 supported though so could send v4 to your backend and V6 to S3

Barry_Based_Benson•8/22/24, 5:27 PM

Does r2 support it?

Barry_Based_Benson•8/22/24, 5:28 PM

Cloudflare uses cf-connecting-ip right

Barry_Based_Benson•8/22/24, 5:29 PM

But would that work with a putobject presigned url

ChaikaOP•8/22/24, 5:29 PM

By S3 API I mean r2s

ChaikaOP•8/22/24, 5:29 PM

You could try and see lol, it might

Barry_Based_Benson•8/22/24, 5:29 PM

I might do tbh

Barry_Based_Benson•8/22/24, 5:29 PM

Thanks

Barry_Based_Benson•8/22/24, 5:55 PM

Does r2 charge for failed requests to the url?

I think it's way more complex then that. A lot of companies use S3 presigned for uploads on websites

Similar Threads

Similar Threads

Similar Threads