Digest storage costs
Hey Novu
We're using the topics feature to send a digested in-app notification to our users. We're still at fairly modest numbers, ~2-3m a day in the activity graph, but with a 1 day digest, this is requiring a rather large storage requirement IMO.
I naively assumed that digesting an activity would roll it up into its parent; but looking in the database it seems like the payload and all the other data is repeated. For some of our more active topics, this might make thousands of such records a day.
Given there is a 1 month expiry, it does seem somewhat wasteful to keep digested notifications in the database, rather than merging the records together somehow or just having a counter on the parent record.
Is there anything that I can do to expedite the process of reaping expired notifications? Or are there better ways to keep storage costs down? I'm weighing up whether to ditch digests and have the clients collate events and send them in one batch; or whether we can run some kind of daily task that finds all the sent notifications and compresses/archives/deletes them somehow.
Any advice? Many thanks in advance
7 Replies
@dmgarland, you just advanced to level 3!
I would set the TTL in the database to be ~1 week but no less then 2 days with the setup that you described.
Thanks @Zac Clifton. How do I do that?
You should be able to go in to the database and on the notification table change the title index
It's not through the app
As in the build-in Mongo feature? https://www.mongodb.com/docs/manual/core/index-ttl/
Hi @dmgarland @Zac Clifton , the TTL (the expireAt index) is configured in the app. It is currently not possible to configure to a different time amount.
My bad, thank you gali!