File Uploading
Trying to implement uploading files for my app. Has anyone tried doing this with wasp? Is there a good way to do this within wasp? Any good libraries?
Everything seems to go through multer, however I don’t see how we can add express middleware through wasp file or pass through headers to actions. Unless I am missing something?
17 Replies
Hey @entjustdoit !
So there are two ways right now to write some code on the server:
1. Via Operations (Actions / Queries)
2. Via app.server.setupFn .
Approach #1 is not flexible enough for now to allow playing with headers.
Approach #2 gives much more flexibility, but it doesn't currently give access to express app, so you can't attach routes to it from there.
Which means we can't do this with Wasp at the moment, although we have plans to add this, and will probably do it very soon, in the following month(s). Here is an issue for it: https://github.com/wasp-lang/wasp/issues/268 .
That said, I had implemented file uploading in the past, in JS / Express / React apps, and you don't really want it to go through your server -> that is a lot of traffic that you rather want to avoid, as it suffocates your server and reduces its capability to do other, more important stuff, like answer user HTTP requests.
Instead, what you ideally want is to upload the file directly from client! So instead of sending it from client to server and then from server to file storage, you want to send it directly to file storage -> skip the server part.
Server still plays its role, but that role becomes doing authentication / giving permissions to client to upload to file storage, instead of doing the upload itself.
I guess the first question is, where do you want to store your files? Ideally you would use some file storage, like S3, or a similar offering from other hosting provider.
Then, you ideally want to find a nice React library that does most of this work for you, and also explains all the details around it. Such library will probably also advise you on how to set up your S3, and what you need to do on the server. And on the server, you will most likely need to implement a route or two that do some authentication with S3 and then provide those tokens to the client (React).
GitHub
Allow defining custom API routes (http) · Issue #268 · wasp-lang/wa...
Right now dev can define operations (actions, queries) which are then consumed from client via RPC that works via http. However, they can't define custom http API roues at the moment! They ...
As for file uploading libraries:
1. I found this one by doing quick search, looks pretty good at first, but then it doesn't have many stars and hasn't had much work on it recently: https://github.com/apideck-samples/file-picker
2. One I used some time ago and that worked well was Fine Uploader: https://github.com/FineUploader/react-fine-uploader -> however I see now it is archived! That sucks.
3. There is this one: https://github.com/odysseyscience/react-s3-uploader -> has ok number of stars, but no work done in the last 2 years. But maybe it is stable?
4. There is this link then, pretty fresh (2022) that covers the process in details, so this might be quite interesting: https://blog.devgenius.io/upload-files-to-amazon-s3-from-a-react-frontend-fbd8f0b26f5 . It might be enough to just follow this.
This would be a cool feature for Wasp -> uploading of files -> and we actually have an issue for it here: https://github.com/wasp-lang/wasp/issues/494 .
GitHub
Ability to serve and upload dynamic binary assets (images, pdfs) · ...
Wasp currently has no way to let users download dynamic binary assets nor upload them. This is a fairly common use case, so should be supported. Where these assets are stored is a related topic (da...
❤️❤️ thank you for the thorough response! Yeah uploading to S3. I initially looked into the option of using signed uploads and skipping my server altogether as mentioned, but all the tutorials I saw kept having security disclaimers which made me nervous.
I primarily wanted to make sure I wasn’t missing anything, and thought since it wasn’t asked before on here would be a good thing to clarify. I’ll dig into those links in the morning, thanks again!
Sure, feel free to ask! This is useful for us also, this helps us refine this topic for the future, so when we implement it we will have a lot of the stuff ready already!
Ah yes, signed uploads, that is the name! Security disclaimers -> I wasn't aware of those, couple of years ago when I was doing it, most of the content online was advising to do signed uploads instead of going through our server. What were some security disclaimers they mentioned?
Btw if you get stuck with anything, feel free to ask for help here, even though it is not Wasp -> as I said, it is good for us to learn about this also.
Didn't go in depth above, but signed uploads caught my eye. I, too, read they were the recommended approach, so curious what security concerns there are. If I remember correct (and it's the same thing, could be diff but I think same), your client asks your server for effectively a time-limited, one-time upload token, your server does direct AWS communication to get it (where it specifies S3 bucket too), gives the client the token, and the client can use to PUT directly to some AWS URL. I felt it was pretty safe that way? 🤷♂️
Yep. Here’s a great little tutorial that shows you how to do it @entjustdoit https://youtu.be/wbNyipJw9rI
TomDoesTech
YouTube
Use Presigned PUT URLs to Easily Upload Files to AWS S3
Theo's Tweet: https://twitter.com/t3dotgg/status/1564019039877271552
Repository: https://github.com/TomDoesTech/pre-signed-put-url
0:00 Intro
0:43 Data flow
1:45 Bootstrap application
2:33 S3 bucket setup
5:07 Get PUT URL handler
10:36 Form
15:24 Does it work?
🌎 Follow me here:
Discord: https://discord.gg/4ae2Esm6P7
Twitter: https://twitter.c...
Implemented and deployed 🚀 thanks for the help!
The security thing was in reference to all the tutorials I was finding and less a comment on the method itself. They kept using fully public buckets for simplifying the tutorials which is what initially turned me off the approach.
awesome! What solution did you end up using?
Ok yeah, that is common pitfall -> buckets need to have proper settings on them. If I remember, you can even put some CORS stuff on them? Not sure though hm.
Yeah, what did you end up with Elis?
@Vinny (@Wasp) @martinsos Pre signed URLs for S3, just made everything more restricted in the permissions panel (eg source on CORS)
wow that was fast! 😄 any way we could make it even easier through Wasp, any ideas?
Hey @entjustdoit any tips or code snippets on how it can be implemented? Will appreciate 🙂
Sure! First, go to S3 and setup a bucket and IAM user, then add these entries to your .env.server file.
then add the aws sdk to your list of dependencies
In your front end, setup the functions for downloading and uploading files.
We now need to create two queries for getting the upload and download signed URLs.
Awesome @entjustdoit , thanks for helping out :D! Ha we have Waspeteers helping each other this is really nice :D.
Simply amazing, can't thank you enough! 🔥🔥🔥
No worries!