Strange Storage usage metrics on the Neon dashboard
Hi all. I'm currently running a free tier account. Since I had a call with a couple of Neon folks yesterday about your new pricing model, I started looking at some metrics present on the dashboard. And noticed something strange about the "Usage since February 23" storage metric : It is currently showing 450 MB but the only branch I'm using is reported on the same dashboard to be 67 MB. I've attached a screenshot to illustrate.
Since the actual data on that branch is around 2 MB for a compressed dump or 10 MB uncompressed. I'm assuming that the 67 MB reported at the branch level represent the full data, plus indexes, plus the 7 days rollback history.
However I don't understand how my monthly storage quota reads 450 MB. Reading https://neon.tech/docs/introduction/usage-metrics did not help me figure out the issue as well.
Can you please help me out here ?
Maybe it's just a glitch and I should stop worrying that I'm approaching that 500 MB free tier quota ? π¨
Cheers ! π
Neon
Usage metrics - Neon Docs
As described in How billing works, each of Neon's plans includes Storage, Compute, and Project usage allowances. The Launch permits extra compute usage. The Scale plans permits extra compute, storage,...
26 Replies
extended-salmonβ’2y ago
I've asked internally because I am curious too
flat-brownOPβ’2y ago
Thank you. FYI I'm now at 571 MB just overnight with very little usage from the users. Do I have to worry that db writes will be suspended ?
extended-salmonβ’2y ago
same here
i ran through calculalating my real storage and got a tenth of the displayed usage
even if the sql function i used is missing somethinge, the fact that you have the same problem leads me to believe that this is a neon issue, hopefully only display wise
extended-salmonβ’2y ago
stormy-goldβ’2y ago
This video should help you better understand your storage consumption: https://www.youtube.com/watch?v=rHgUnKHRa2A
Long story short, the storage size that you see in the UI correspond on the size of all your branches + the history of changes which happened during the period covered by your history retention period.
Neon
YouTube
Understanding Your Storage Costs
In this talk, Anastasia Lubennikova will explain how to understand your Neon storage costs and how you can optimize them.
You can learn more about Neon's pricing in our documentation: https://neon.tech/docs/introduction/billing
β‘οΈ Try Neon at https://neon.tech
β‘οΈ Follow us on Twitter https://twitter.com/neondatabase
β‘οΈ Star us on GitHub https:...
stormy-goldβ’2y ago
In this specific scenario, reducing the history retention would help reducing the storage usage
extended-salmonβ’2y ago
History retention only has 1 or 0 days, given your answer I was kind of expecting something different... I'll have to watch the video first, but logically i still have issues with where the *10 comes from...
flat-brownOPβ’2y ago
hi @Yanic . Does the size displayed at the branch level (on top of the total storage used) includes the history retention for that particular branch ? I am assuming that it is the case because the data is around 10 MB but the UI displays 67 MB for the branch.
extended-salmonβ’2y ago
Ive watched the documentation and turned off the change saving
went from 450 MiB to 134, still double what Im expecting but better
stormy-goldβ’2y ago
@Adit Yes it does π
Very quick demonstration:
stormy-goldβ’2y ago
I ran a couple of :
stormy-goldβ’2y ago
and....
stormy-goldβ’2y ago
π
flat-brownOPβ’2y ago
thank you this is helpful π
flat-brownOPβ’2y ago
but then why is my single branch (main) only 67 MB including history and the full data usage is still 570+ MB ?
flat-brownOPβ’2y ago
BTW : I never had another branch
stormy-goldβ’2y ago
You raced me :p I was about to ask if you had any other branch π
Can you give me your project name please? I can have a look for you
flat-brownOPβ’2y ago
sure I'll PM it to you if you don't mind
stormy-goldβ’2y ago
DM is fine yes, thank you π
extended-salmonβ’2y ago
I went from 450 MiB to 130 MiB by turning off change persistance (from 1 to 0 days) as I said still more than expected, but I see how that can easily mulitply data usage
I also only have one branch
stormy-goldβ’2y ago
@xewn Can you please DM me your project name? I'm happy to have a look for you π
extended-salmonβ’2y ago
will do
stormy-goldβ’2y ago
(Just to be completely transparent should anyone else find this thread: I noticed an anomaly in our Admin UI. It seems that the retention period is still set to 7 days, the previous free tier limit, instead of 1 days as displayed in the UI. I reported this issue to our engineering team and I'm awaiting for a feedback. I will update this thread once I have a confirmation)
This was discussed with our engineering team who confirms that there is a misconfiguration here.
For some projects, the history retention is still set to 7 days instead of 1 day.
This artificially increase the storage consumption for the impacted accounts.
We're currently looking at correcting this misconfiguration for all impacted projects.
Should you want to correct this issue immediately, you can change the history retention settings to 0 and then back to 1 day. This will reset the settings currently in place and should immediately reduce your storage consumption.
like-goldβ’2y ago
Hi @Yanic I've been having a similar issue as the others reported. However, I only inserted about 40 to 60 rows (total since I started). The rows did have Vector data [vector(384)] data. I had only joined Neon for a day by then and was already showing usage close to 400-500 MiB. An individual table with 19 rows is showing 40KB, as a benchmark/example, I'm now 2 days into my Neon account and haven't done that much more and storage is even more maxed out. I mentioned this in the General channel, but was looking for more info and found this thread. It says I'm at 743 MiB after 2 days with less than 100 rows ever inserted. I did do some Prisma migrations, but I'm still trying to account for the 700MiB. I'm concerned about hitting caps in production, or getting charged for so much more than I really use if I go paid.
like-goldβ’2y ago
I'm thinking this is a bug based on convo in General, as I've seen other storage numbers go down in my account without me doing anything. Just concerned about the uncertainty of caps (for free) and charges (for paid) for a production account. Any input you have would be appreciated.
stormy-goldβ’2y ago
Hi @mike, thanks for reaching out. Can you please ping me either your email address, the project name or the endpoint name in DM?
I would be happy to have a look for you π