Unhealthy project status
@supabase my projects is having many unhealthy status since 5 minutes ago, is there any breaking changes ?
203 Replies
experiencing the same thing
Same
Hello, same thing, we can't login
Looks like some services might be down/ I also have the same issue in my projects. I will report to the Supabase team now.
I have the same thing, both my prod and staging environments are down
any updates?
For me too can't connect to my websites
Same here
Us too
We keep having issues pretty often lately, could status page be updated more often and not have to check community posts to find issues
Google SSO is not working for me!
My application uses google sso and its giving me 556
Same for us
Same here
same for me too
{ "code": 403, "errors": [ { "domain": "usageLimits", "message": "Quota exceeded: Your project_and_region exceeded quota for max number of jobs that can be queued per project. For more information, see https://cloud.google.com/bigquery/docs/troubleshoot-quotas", "reason": "quotaExceeded" } ], "message": "Quota exceeded: Your project_and_region exceeded quota for max number of jobs that can be queued per project. For more information, see https://cloud.google.com/bigquery/docs/troubleshoot-quotas", "status": "PERMISSION_DENIED" }what i saw in postgrest errors
my project's dead
:supafire:
Where can I find out when its back on?
There are too many bugs too often; my employer and his team can no longer work. Supabase, which I was the first to defend, is starting to annoy me.
is any Engineer from supabase working on the issue?
Yes I agree, we have lots of issues lately and app is down during peak hours
I am getting CORS problems when trying to login - is that reltaed to this issue?
I have the same issue
Just joined because I am experiencing the same issue on multiple projects.
yes, auth is down + postgrest seeingly having issues
same issue here
There is a current outage, I am sure they are working to resolve it, I understand it affects all of our projects, but the team will be working as hard as they can to resolve it as fast as possible
same issue here
why the hell is the service page now honest about this being a full outage
Same, auth and postgre are both down for me
hope this fix soon. wew
it's just a simple management api issue
If it's google sso limit problem, it can be quick to resolve
coming here to see am I not the only one, I thought it was related to pgjwt but looks like it is down?
there's a world outage aatm
Getting CORS issues
I even saw Whispr flow to use supabase and to be down, this will be a costly day
experiencing the same
hello, so nothing to do right? auth not working in my proyects
our app is down
same
my employer is calling me none stop whats happening, when can we expect a fix.
Getting CORS issues too
we had cloudflare, now supabase? π€¦ββοΈ
our app is down
I just lost the most important demo of my life
can someone vibe this rn.
Stop spamming useless cry messages please
We don't know when a fix will arrive, we just have to be patient, everything will be restored as soon as possible
Please fix fast
:L_:
WHY NO UPDATES ON STATUS PAGE? STAFF?
At least be transparent, so that we can share info to our customers. Why making this invisible from the public?
been so unreliable this year its crazy
I bet most of you are free users complaining?
No pro userβ¦
No, paying customers here.
Not really
on god fr fr I had to come to Discord to verify while working
Updates will come as soon as they can, their priority is fixing the issues
enterprise lol
I'm on pro
stfu baba, i'm paying 200$ per project
:lick:
Pro customer here, auth down
New Fireship video incoming π
paid customer, both their js and py clients seem to be down.
That escalated quickly
Paying 2k a month
same issue here unhealthy auth
Where is staff?
Me too!
Pretty sure the person responsible for tweeting acknowledgement of the issue is not going to be the one resolving it..
worst status page tbh
if we send more gifs maybe the issue is resolved faster
Is Supabase down?
why tf is everything on green
put a normal status
tf
Unless you missed my message above or the status page update. I have also pinned this post. So Supabase is aware of the issue and are looking into it.
Still, we need an issue where we can actually find any progress
who is in charge of this status page
I was trying to solve for half an hour
fix fast please
phil me too man
because your status page ain't doing a normal status
Make it public, we need to share it with our customers.
Why are you guys competing who has the most supabase bills
status pages need to be automated and HONEST
if there are services down on 90% of the projects then why not AUTOMATE that and SHOW
Status page is showing an issue at the moment
Please read my message properly. The status page was updated as soon as I reported the issue.
baba, it's showing something as a 'minor' issue
Guys calm down jesus christ π
bro this issue could be anything
yeah for management which is like correct but far from the full issue
'some management api issues'
Fair enough, and I don't disagree, but, as with all things, the eng's will be the notified, however, the social media manager probably won't, Supabase is a US based company, it is like 1am there
supabase trying to hide their status
you need to join the discord to get the right state
Yes and that is the current situation maybe?
@Dinuka clearly you're not working in a prod environment
great they are working on issue, take 2 sec and give an accurate status page...this is no management api issue
kinda dumb aint even gonna lie
Haha bro I am. But this doesnt help the staff π
auth down, db down, everything on red
err [AuthUnknownError: Unexpected token 'I', "Internal s"... is not valid JSON] {
__isAuthError: true,
name: 'AuthUnknownError',
status: undefined,
code: undefined,
originalError: [SyntaxError: Unexpected token 'I', "Internal s"... is not valid JSON]
} { user: null, session: null }
{
"code": 403,
"errors": [
{
"domain": "usageLimits",
"message": "Quota exceeded: Your project_and_region exceeded quota for max number of jobs that can be queued per project. For more information, see https://cloud.google.com/bigquery/docs/troubleshoot-quotas",
"reason": "quotaExceeded"
}
],
"message": "Quota exceeded: Your project_and_region exceeded quota for max number of jobs that can be queued per project. For more information, see https://cloud.google.com/bigquery/docs/troubleshoot-quotas",
"status": "PERMISSION_DENIED"
}
whats the isus guys? You forgot to pay to google.?:)
this aint chatgpt
"Unresolved incident: We are investigating reports of issues with our Management API." is not clear enough.
Imagine your customers customers would behave like you π
I think they first have to investigate to know the exact issue before they can say more.
they added in status now
lmao good bait
yeah thats exactly what happened.
thats not the issue though that left over from their weekend maintenance...that went well
I can relate to it, but please understand that we use Supabase as a data component for a major FinTech client in Asia, and this announcement and response for incidents are totally unacceptable.
hey all of our users are getting 401
What would you rather them say?
fix fast
last time the page wasn't updated for hours before acknowledging an issue
quick someone ask Claude to fix it
just pressing alt+f4 will fix it
At least supabase is already aware of this
the status page dev is not working on mondays
It's finally works for mr
working for me now
I'm getting 500 internal server for the API, is that the same as the management API being down?
I'm fairly new to deving
Same for me. My dashboard shows the following error - even when I host on AWS!
Fixed, at least storage loaded for me again
its recovering already
somoene forgot to pay their GCP bill
Something like "Authentication service is failing to serve requests" so our client won't be worried about the data loss but just to be clear that only auth is affected.
Anyone aware of a good way to host a mirrored version, for fallback?
i.e. Supabase Hosted is primary, AWS is secondary
Aware of database replication, but wondering how you'd approach auth, postREST, client-side connection...
it works now
calm down ! Take easy ...rsrs
Internal server error. supabase
auth is not working
looks like it's fixed
Working for me now
Working for me now
glad i got the email about their upcoming event at 3am, maybe an email saying our service is down
It working again for me
working
It is recovering
Does supabase db hosted on GCP?
working now
Nice RTO devs
@angry people, these can all be true π
* This sucks for you
* This sucks for Supabase
* This shouldn't hapen
* Safe to say Supabase employees are doing everything rn to fix this
Would love to hear if anyone have experience on this one. From what I see the problem relies on the upper layer of db (PostgREST and GoTrue). It should be easy to swap that out to the self hosted one
I don't think it's just the auth service having issues as my dashboard is showing other services offline too.
thanks everyone for the moral support here.
This! π― Thanks for fixing it fast team
Thanks for quick fixes
so what was the issue exactly?
google quata! big query!
same here
Can I sue for SLA compensation
Please report to the team that a root cause analysis is demanded
after AWS, CloudFlare now my SaaS was of because of Supabase
this issue was faced by pro users as well?
Yes, it was a global outage
You can open an support ticket asking for that.
seems to be back, thanks for the fast fix Supabase
you're a big corp you should give that without asking
It seems everything works now
Why not just be transparent?
Thanks for fixing the problem.
I'm not a Supabase staff.
My site is up and running!
Thanks Supabase team for the speedy recovery!
This is why you should request this from Supabase. I'm not a Supabase staff. I just moderate their discord.
Cheers for sorting super quickly!
Any advice on where to look for safely replicating supabase on AWS? (not just DB, but whole service)
loudworks!
then what is the use of the discord if they don't get to understand what users actually are asking for?
Works now - thanks guys
the database is just postgres
Sadly thats not so easy. Took us weeks to get it running properly on Kubernetes
Thank you, but I'd say this is the shame on Supabase to rely on volunteer handling all these stuff.
You could follow one of the self-hosting guides https://supabase.com/docs/guides/self-hosting
Self-Hosting | Supabase Docs
Host Supabase on your own infrastructure.
You can check about self-hosting supabase, its possible but not very documented very well
fix fast
They have official channels which you can reach. I am compensated by Supabase for my work here, so it's not free.
Thanks for RE! Can imagine it's non-trivial, any top-of-the-head words of wisdom? Thank you btw!
It's for users to help users. They have official support channels which you can reach out to them on. Discord is not it.
stupid strategy
Ok sir
https://supabase.com/docs/guides/self-hosting Is a great start, suppose the tricky bit is that smart routing... anyone had experience here?
run it with coolify one click deploy
Supabase:
I dont know if related to this recent issue but now when i try to PUT something to Supabase im recieving in all requests RLS error
κ° μκΈ°λ€ μ΄κ±° γ
γ
γ
γ
γ
System started to look green again, thx for quick fix supabase team
it's still down?
@romulus @Lee - Supabase team hasn't confirmed a final fix. Rollout is currently happening with a fix included, but auth is distributed, so sometimes it'll work, sometimes it won't.
Auth layer performance will be rocky until the status page changes:
https://status.supabase.com/incidents/sh7gmmyktjzw
Supabase I love you, but this breakdown of everything to them it's scary, especially since I use your services for all client and personal projects, I have a webmaster for adult content, my visitors must have been sad for a moment
It seems services are back to normal so unpinning this post.