s3 uploads (but im local?)
I’m developing a ComfyUI cloud workflow hosted on a RunPod Serverless worker. (just think cloud gpu)
The worker supports automatic uploads to S3-compatible endpoints.
In production, I’ll use a proper cloud bucket, but during development I’m running Supabase locally via supabase start.
Local setup:
API URL: http://127.0.0.1:54321
GraphQL URL: http://127.0.0.1:54321/graphql/v1
S3 Storage URL: http://127.0.0.1:54321/storage/v1/s3
DB URL: postgresql://postgres:postgres@127.0.0.1:54322/postgres
Studio URL: http://127.0.0.1:54323
Issue:
The RunPod worker runs in the cloud, so it obviously cant reach my local Supabase’s S3 endpoint (127.0.0.1:54321).
Goal:
I want to simulate production-like S3 uploads from the RunPod worker while keeping Supabase running locally during development.
Does anybody have any idea how I can do this? I know the easy way would be to just move onto the cloud free tier, but I am not at that stage of project yet - I really just need to test the code for uploading to bucket
I also believe its not as simple as just uploading to an external cloud bucket because im doing things like signed url uploads scoped on users jwt etc
16 Replies
I guess to simplify the question - how do signed URLs for upload work when developing locally?
I was thinking this, but then would I not need to sign the URL with this tunneled URL?
If this works may be the easiest solution
For now I just need something temporary so I can get some code working on the serverless which actually does the logic of uploading to my signed url etc
i.e. takes in the upload url and puts the assets in that bucket -
so..
1) create tunnel
2) create + sign upload URL using the new tunnel URL i have
3) Pass in signed 'tunnel' url
?
You have 2 options
- in your
createClient call use this tunnel url directly, so now signed urls will automatically have this url as domain
- or replace the localhost in signed url manually or with a scriptOh wait! I think I see what you mean!! So literally in my .env.local
I can just replace this URL with the tunnelled version and it all should flow through with the minimal code changes right!
I think this makes sense !
Yes. I think this should work
For example, make a GET request to this url
This is proxied to my local instance
Got youuuuuu!!!! Okay let me give this a go (Im a new developer so this was slightly worrying to me, but i think i see what you mean)
and then the tunnel basically points to the localhost anyway
legend
thank you ❤️
One quick question, doing it this way would result in em having to update many areas such as google auth callback endpoinst etc right?
whereas if I go down the other approach you suggested where I just replace the localhost in signed URL manually
I could just have this as a small replace function sitting on my serverless endpoint and that pretty much simulates a production flow
i.e. if its ever localhost, replace it with my proxy URL and then send it to that, then my proxy / tunnel just sends it through to my local
Correct. So in your case it will be better to simply replace localhost:54321
LEGEND
okay thanks so much man 🙂
Its nice to get confirmation from someone with experience that this will work before I spend days trying to figure it out, so this is good 😄
I just need to make sure that this works with signed upload url's as well. I've tested with every other supabase service(those work) execpt this one so let me know if it doesn't
It works, I just tested it. Also you only need token from the result so you don't even have to replace the localhost
wait, little confused by that.. so when you say I dont need to replace localhost...
What?
So just create a tunnel that exposes the local ? or I dont even need that?
because if i dont need to replcae localhost then i wouldnt need a tunnell would i
When you use method createSignedUploadUrl it also returns a token
And you use this token in uploadToSignedUrl method
with open("./public/avatar1.png", "rb") as f:
response = (
supabase.storage
.from_("avatars")
.upload_to_signed_url(
path="folder/cat.jpg",
token="token-from-create_signed_upload_url",
file=f,
)
)
yes like this right?
but are you saying that we dont need to replace localhost meaning we dont even need the tunnel?
I might be being stupid, im sure when i get to the implementation ill realise waht yo mean
No i meant you don't need to replace localhost in the signed url which is returned from the method
Also about the code
When you create a signed url, log the response
The code you have shared is for uploading to signed url
https://supabase.com/docs/reference/python/storage-from-createsigneduploadurl
Click on the response, it also returns a token. So you use this token in uploadToSignedUrl