Import from SQL to Supabase
Hello there! I have a SQL Server database that is modified every three months (some rows are erased). So far I ran a query in SQL, downloaded and stored the Excel files (no more than 5 MB each, in total 20 MB annually) but I don't want to make it manually.
I need a way to automatically import the original data into supabase. I want to query the last three months of data, and add it to the online database automatically (some kind of Cron job or something similar).
What would be the easiest option? Thanks in advance! 🙂
11 Replies
You can do that inside postgress instance
Thanks @tonyhart may you please let me know how to do it? Is there any tutorial or resource? Thanks in advance
the easiest way I can think is query your data to excel and make it csv
then inside pgadmin you can import that to database
How To Migrate a MySQL Database to PostgreSQL Using pgLoader | Dig...
The prospect of migrating a database can be intimidating, especially when migrating from one database management system to another. This tutorial provides in…
Can it be done online instead of manual? Also, the data is stored in a MS SQL Server, not MySQL. Would this be a problem? Thanks again!
the concept is same
@xerox_04 use airbyte
Thank you very much! 🙂 Seems nice to try!!
@xerox_04 depending on your org's security requirements, you can use any one of a number of ETL tools.
Fivetran is King, but probably too expensive for your needs.
Airbyte is an open-source platform (also offers a cloud service, though) that you can deploy on your own machine, or as a docker image on AWS/Heroku/DigitalOcean, or (my preferred method) via Plural https://www.plural.sh/
Fivetran is King, but probably too expensive for your needs.
Airbyte is an open-source platform (also offers a cloud service, though) that you can deploy on your own machine, or as a docker image on AWS/Heroku/DigitalOcean, or (my preferred method) via Plural https://www.plural.sh/
Plural | Deploy open-source software on Kubernetes.
An open-source platform to build, maintain, and scale infrastructure on Kubernetes. Batteries included.
I'm not a paid shill for Plural but the platform has seriously improved my life and I'm trying to spread the good word. here's some instructions if you're interested -
1. Create an account at Plural 2. Create a new project @ Plural. You'll need an API credentials for AWS or GCP or Azure. You'll also need a Github account. 3. Plural will handle all of the technical stuff. You install Airbyte on top of Plural - its as simple as installing an app via Apple's app store 4. Login to your new Airbyte instance.
5. Create a newSourcewith your credentials for MSSQL Server 6. Create a newDestination- Choose Postgres and then plugin your supabase credentials. 7. Choose your syncing Schedule - in the case you've described,manualis probably the best option 8. Whenever you get an update to your dataset, login to airbyte, click Sync Now and forget about it
Thanks for such detailed explanation! I'll for sure give it a try!!!