Digest storage costs
Hey Novu
We're using the topics feature to send a digested in-app notification to our users. We're still at fairly modest numbers, ~2-3m a day in the activity graph, but with a 1 day digest, this is requiring a rather large storage requirement IMO.
I naively assumed that digesting an activity would roll it up into its parent; but looking in the database it seems like the payload and all the other data is repeated. For some of our more active topics, this might make thousands of such records a day.
Given there is a 1 month expiry, it does seem somewhat wasteful to keep digested notifications in the database, rather than merging the records together somehow or just having a counter on the parent record.
Is there anything that I can do to expedite the process of reaping expired notifications? Or are there better ways to keep storage costs down? I'm weighing up whether to ditch digests and have the clients collate events and send them in one batch; or whether we can run some kind of daily task that finds all the sent notifications and compresses/archives/deletes them somehow.
Any advice? Many thanks in advance
We're using the topics feature to send a digested in-app notification to our users. We're still at fairly modest numbers, ~2-3m a day in the activity graph, but with a 1 day digest, this is requiring a rather large storage requirement IMO.
I naively assumed that digesting an activity would roll it up into its parent; but looking in the database it seems like the payload and all the other data is repeated. For some of our more active topics, this might make thousands of such records a day.
Given there is a 1 month expiry, it does seem somewhat wasteful to keep digested notifications in the database, rather than merging the records together somehow or just having a counter on the parent record.
Is there anything that I can do to expedite the process of reaping expired notifications? Or are there better ways to keep storage costs down? I'm weighing up whether to ditch digests and have the clients collate events and send them in one batch; or whether we can run some kind of daily task that finds all the sent notifications and compresses/archives/deletes them somehow.
Any advice? Many thanks in advance
