Topics

Supabase•5mo ago

Elad Ben-Israel

Any known issue with realtime database updates?

For some reason we are not getting all the realtime notifications for database inserts in our clients (we see the row added to the table, but clients don't receive a notification). Anyone seeing this issue?

38 Replies

mr nooli•5mo ago

Yes we are. For us it's working perfectly (I think) on dev, which is a lower tier instance, but not working at all on production. I was under the assumption that it was caused by a DNS switch from Cloudflare to Vercel (the main app hosting) but it should have 0 impact on Supabase's Realtime functionality. Having severly degraded operations at the moment due to this. For some more debug info incase it helps you @Elad Ben-Israel - I'd recommend combing through the realtime publications to ensure that all the tables that need it have realtime enabled. Check RLS policies, and test them via the dashboard Double check table names from the client and the filters that are being applied. Finally, try this:

.on('postgres_changes', {
                event: '*',
                schema: 'public',
            }, (payload) => {
                console.log("DB_CHANGES", payload);
            });

.on('postgres_changes', {
                event: '*',
                schema: 'public',
            }, (payload) => {
                console.log("DB_CHANGES", payload);
            });

For me this wasn't working and that's when I knew that it was an issue with the supabase instance. Working locally (via the dev instance) and on dev, but fails to work locally (when connected to prod instance) and in prod.

Elad Ben-IsraelOP•5mo ago

Thanks @mr nooli - this helps Let's see if the supabase team can chime in. This is also degrading our operations

mr nooli•5mo ago

Agreed - however I don’t know if supabase team monitors this. I’d recommend putting in a support ticket via the dashboard. (I did last night)

Elad Ben-IsraelOP•5mo ago

More details - we are observing about 70% message loss (we are getting realtime notification for 30% of the INSERT operations).

mr nooli•5mo ago

For us we’re not getting any updates, but every so often one or two slip through, roughly 90% message loss. For all operations

filipecabaco•5mo ago

hi, realtime dev here, do open an issue so I can check your project

mr nooli•5mo ago

I have done - 24860690221

filipecabaco•5mo ago

👍 checking

mr nooli•5mo ago

Feel free to dm me directly if you need more details!

Elad Ben-IsraelOP•5mo ago

Here is our support ticket 24819616879

filipecabaco•5mo ago

@mr nooli you should see a more stable connection now. I would still advice to move to broadcast from database as it's more stable @Elad Ben-Israel checking yours now

Elad Ben-IsraelOP•5mo ago

Can you elaborate what you mean by broadcast?

mr nooli•5mo ago

Also can I ask, what was the issue?? Is it something that I could’ve prevented?? Implementation details wise, what changes?

filipecabaco•5mo ago

we have a new system that makes it more stable https://supabase.com/docs/guides/realtime/subscribing-to-database-changes

Subscribing to Database Changes | Supabase Docs

Listen to database changes in real-time from your website or application.

filipecabaco•5mo ago

postgres changes works with a polling query which means that it's more prone to error and it's more limiting scalability and stability wise

Elad Ben-IsraelOP•5mo ago

Got it. we will look into this Seems like things are working now

filipecabaco•5mo ago

broadcast from database works differently as it instantly pushes from a process that is listening to a replication slot and pushes the update ASAP cool I had to restart both nodes the underlying issue is on our side @mr nooli

Elad Ben-IsraelOP•5mo ago

Thanks @filipecabaco for the quick response. Much appriciated.

filipecabaco•5mo ago

we are using and eventually consistent lib to persist an unique connection against the database which means that, eventually consistenly, seems to not solve the conflict properly

mr nooli•5mo ago

I tried restarting the project yesterday 🤷‍♂️

filipecabaco•5mo ago

we're working as I speak in fixes and mitigates *mitigations

Elad Ben-IsraelOP•5mo ago

Great

filipecabaco•5mo ago

meaning that: our main focus is not new features but improved stability

mr nooli•5mo ago

That’s fantastic thank you! I think it would be beneficial to get pros/cons for the two different types of db updates.

filipecabaco•5mo ago

with that in mind broadcast from database already offers way better stability honestly the biggest con of the new system is that, to have a similar experience to postgres changes, you require to do filters on the trigger and to set up RLS for realtime

mr nooli•5mo ago

As far as I can see there’s quite a lot to setup the new system for db updates.

filipecabaco•5mo ago

but it's not mandatory if you use realtime.send yes it still is a lot of setup ... we want to stream line it all we will include more guides, UI elements and focus on having a similar experience we also want to implement broadcast filters (e.g. instead of just being filters for PG changes they are applied to messages broadcasted from the server) but that is still TBD and no ETA as new features won't do much with an unstable service 😅 but for clarity: the future of database changes being sent to client will fall more under the broadcast from database approach it has shown better results scalability and stability wise in fact it was the only way to unblock certain customers that required 4000+ concurrent users which was breaking postgres changes finally, big big sorry for the issues caused :/

mr nooli•5mo ago

That sounds good, right now we haven’t setup RLS for each table as we work via prisma. The setup for “who can see these updates” is quite complicated - so is it a “must” for the new system?

Elad Ben-IsraelOP•5mo ago

no worries. this stuff happens. really appriciate the responsiveness

filipecabaco•5mo ago

not a must just extremely useful

mr nooli•5mo ago

I appreciate the speedy response @filipecabaco !

filipecabaco•5mo ago

you can use realtime.send without any policies also, for realtime authorization you can set only rls policies against realtime.messages

filipecabaco•5mo ago

🙏 thank you both for understanding and we will keep improving if you want to track our work you can here https://github.com/supabase/realtime/pulls

GitHub

Pull requests · supabase/realtime

Broadcast, Presence, and Postgres Changes via WebSockets - Pull requests · supabase/realtime

filipecabaco•5mo ago

the advantage of being opensource 😆

mr nooli•4mo ago

I believe the same thing is happening for my project again!

Elad Ben-IsraelOP•4mo ago

@filipecabacohi there. we are experiencing a similar issue today with the realtime updates Are you guys aware of this?

filipecabaco•4mo ago

do open a ticket so we can check your instance

SwagSupreme•3mo ago

We are experiencing major issues with realtime not sending messages in production. Possibly related is that websocket connections are being closed and reopened (we have observed this) but the realtime client does not resubscribe to any previous channels. Support ticket: SU-217068

Supabase gives you the tools, documentation, and community that makes managing databases, authentication, and backend infrastructure a lot less overwhelming.

41KMembers

View on Discord

Did you find this page helpful?