Database is 100% down, low disk space. Cannot upgrade disk space due to cooldown
Our database is down and 100% unresponsive and not able to connect to or see metadata in the Supabase dashboard. I think it's because we ran out of disk space, but I cannot increase disk space because we are in a disk space cooldown.
Can we reset the cooldown to add more disk space to the DB?
4 Replies
Sort of surprised disk space would stop the dashboard settings/usage tabs.
Did you just go thru a disk size upgrade? If so the docs say there is no avoiding the cooldown as it is an AWS thing.
Yes, we did an upgrade, and then were trying to delete old rows, but weren't vacuuming frequently enough and it sucked up more DB space
Our DB cleanup SQL command errored and exited with this:
psql:db/cmds/drop_old_dbos.sql:30: WARNING: terminating connection because of crash of another server process DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory. HINT: In a moment you should be able to reconnect to the database and repeat your command. psql:db/cmds/drop_old_dbos.sql:30: FATAL: DbHandler exited psql:db/cmds/drop_old_dbos.sql:30: SSL connection has been closed unexpectedly psql:db/cmds/drop_old_dbos.sql:30: error: connection to server was lost
Sorry, beyond my skill set as to what that means.
I would have in a support request, but it is the weekend...
But if your disk size upgraded I don't think they can force an upgrade until the cool down is over.
https://supabase.com/docs/guides/platform/database-size#reducing-disk-size
You probably have gone thru that.
Yeah, read it. Thanks for sharing anyways though.
We are going to fail over to a different DB project and run on that until we can fix our prod db