Since days i am searching for a way to mass insert/upsert data from a file stored in supabase storage bucket. I tried creating a edge function thag triggers when a new file appears in the bucket, then i tried to insert the data but it always ends with the error: memory/cpu limit exceded
I tried different file types to and background tasks, all the same (File types: csv, json, ndjson) file size is from 20-30mb
The SQl COPY command also doesnt work because it wont work with supabase storage somehow.
Also tried it with functions but that didnt work either.
I need to import from a file because they get auto uploaded from my customers Windows Servers.
Has anyone an idea how to handle this? Why is there no solution to this already for bulk inserting?
Would be great to have the csv inserting function available in the dashboard as a js function