R
Railway13mo ago
Coates

Postgresql, only getting 512 mb memory on hobby plan

hello, looking for tips for my postgresql service, seem to be capped at 512 mb memory on hobby plan tried restarting but still only hitting 512
No description
Solution:
there's a volume now, this is the new database as a service, new postgres version, new supporting infrastructure
Jump to solution
33 Replies
Percy
Percy13mo ago
Project ID: 2d8142fa-1d81-44b3-bd4d-c4bb310dd3c3
Coates
Coates13mo ago
2d8142fa-1d81-44b3-bd4d-c4bb310dd3c3
Brody
Brody13mo ago
why do you want it to use more memory? is it crashing?
Coates
Coates13mo ago
I was getting an out of memory issues in python when doing some large batch insertions, but i've since reduced the number of inserts to avoid the error i was receiving. i expect having access to more memory would improve performance when inserting rows and also not slow down others accessing the database
Brody
Brody13mo ago
did you get these out of memory error logs from python or from the postgres service to be clear, at this time I don't think postgres is limited to 512mb, I think that's just simply what it's currently using
Coates
Coates13mo ago
python only, trying to find the error i received but when googling a lot of feedback around memory server side, i.e. killed query. let me see if i can find the error i was receiving SSL SYSCALL error: EOF detected was the error
Brody
Brody13mo ago
logged from the python service, correct?
Coates
Coates13mo ago
correct was returned from within debug menu in python tried to check logs in railway but was chugging, going to see if i can find that when it happened
Brody
Brody13mo ago
what client are you using in python
Coates
Coates13mo ago
vs code
Brody
Brody13mo ago
that's a code editor I'm asking what client library you are using in python
Coates
Coates13mo ago
oh sorry psycopg2
Brody
Brody13mo ago
read this https://blog.stigok.com/2021/02/28/sqlalchemy-postgres-ssl-eof-detected.html it's not an issue with railway or with the postgres database, it's a code issue
Coates
Coates13mo ago
thanks for the dialog. i will see what i can do to implement a pre-ping like behavior and see if that allows the larger insert. i am not using sql alchemy if thats helpful at all. only thing I will add is i've been able to reproduce by trying to insert a larger number of rows, and see the metrics in railway only hitting exactly 512 during these inserts. not sure what memory settings within postgre are set by default in the railway image
Brody
Brody13mo ago
I've seen plenty of postgres databases use well over 512mb of memory, I'm confident this is just a code issue
Coates
Coates13mo ago
I would normally expect to see some variation in memory usage, i.e. above and below 512 when performing operations , especially if i increase the number of transactions that are ocurring seems like a disconnect occurring mid query rather than before the insert op occurred as described in the blog you posted
Brody
Brody13mo ago
from your screenshot it looks like it's going from 300 to 500mb does this database have a volume attached
Coates
Coates13mo ago
it is going from around 300 to exactly 512, but never above. the 300 memory is when no operation is taking place (in between batch insertions)
No description
Coates
Coates13mo ago
i am not sure , i did not create any seperate services/storage options when creating the service within railway
No description
Brody
Brody13mo ago
oh that's the old database, I was under the impression that you where using a new database even so, where your python service?
Coates
Coates13mo ago
python service is on my local machine
Brody
Brody13mo ago
gotcha
Coates
Coates13mo ago
or rather, python script just doing some testing stuff nothing actually being used
Brody
Brody13mo ago
you are currently using a deprecated database plugin, plenty of issues with those. go ahead and deploy a new postgres database
Coates
Coates13mo ago
clicked add, seems the same?
No description
Coates
Coates13mo ago
oh i see it has a disk?
Solution
Brody
Brody13mo ago
there's a volume now, this is the new database as a service, new postgres version, new supporting infrastructure
Coates
Coates13mo ago
interesting.. how did i manage to create the old one?
Brody
Brody13mo ago
you created it before the new databases went into general availability so my bad, shouldn't have assumed you where using a database v2
Coates
Coates13mo ago
thanks brody, i will try with the new v2 tomorrow.. interesting though i created on 10/23, did v2 come out since then? appreciate your time tonight as well
Brody
Brody13mo ago
I'd have to check when when database v2's went into general availability
Coates
Coates13mo ago
thank you again! will let you know how it goes
Brody
Brody13mo ago
sounds good!
Want results from more Discord servers?
Add your server