Presigned URLs (S3 upload): file type verification, image transformation & cleanup/delete
When implementing S3 upload, a lot of examples I've found (and also Theo's recommendation afaik) are using presigned URLs.
Maybe I'm missing something, but I have three questions about this:
1. Verify file type
S3 allows to restrict the "content type", so e.g. only jpg or png. But it does not actually check the file itself, only the extension (as part of the file name). Users can upload .png files that are actually .txt files, just by renaming the file. You cannot check the MIME type without a server inbetween.
How do you verify the file type?
2. Image transformation
Don't you care about the image size? You can only reliably convert the size on the server, which obviously does not work with presigned URLs. I know you can restrict the image size via presigned URLs, but don't you want to decrease the file size (of e.g. a jpg) to save some costs of the storage?
3. Cleanup old images
When a users "replaces" an image (imagine an avatar), you need to delete the old image as well. This can only be done reliably on the server as well if you want to make sure that "upload" and "delete" is done in one operation (the API request).
I guess one could also model the storage path via a fixed ID, so you just overwrite the image in the storage, but it sometimes you need random file names, no?
In general: Is really everyone using presigned URLs?
If yes, how do you overcome the three mentioned topics?
2 Replies
have you found a solution yet?
@jannis.gg Yes and no, I implemented some workarounds like so:
1. Verify file type
When a user uploads a file, I also save the file extension in the DB. So when I fetch the image, I include the extension manually for the path. But I still have no way to verify the file type for the MIME types.
2. Image transformation
Not really. Maybe I have to implement some kind of webhook to inform the API about a completed upload that itself triggers a GitHub action or so. I think Uploadthing's "onUploadComplete" does something similar. But I haven't found time to do so.
3. Cleanup old images
I now actually save the images in specific S3 folders. The path is the image ID I also save in the DB (e.g. /avatars/user123/my-image.png). So when a user updates his avatar, I first use the AWS SDK to delete the whole folder (here: /avatars/user123/), that way I don't have to remember old/other folder paths. This implicitly means that there can only be ONE image per user.
What exactly do you mean with "look at the file contents"? AFAIK you can only check the MIME type by "downloading" the file, which we don't want to do in the serverless function.
Please let me know if you have any more info on this!