Novu•2y ago

Redis connection is getting stale on idle

We self hosted Novu on Azure and we are currently on version 20. We are using Azure Redis and noticed that if the worker app is idle, the redis connection is getting stale and have to restart the worker to process the messages. Is there a setting to keep the redis connection alive even though there is no traffic?

Uujwal1373 We self hosted Novu on Azure and we are currently on version 20. We are using Az...

Zac Clifton•1/18/24, 10:22 PM

After how long do you notice it going stale?

Zac Clifton•1/18/24, 10:24 PM

Also it looks like you could set the REDIS_KEEP_ALIVE env variable to set this to a lower number
https://github.com/novuhq/novu/blob/e888590b9204c6ff3b8319c184dfcbc9003ddf80/packages/application-generic/src/services/in-memory-provider/providers/redis-provider.ts#L22

GitHub

novu/packages/application-generic/src/services/in-memory-provider/p...

The open-source notification infrastructure with fully functional embedded notification center

novuhq/novu

Zac Clifton•1/18/24, 10:24 PM

assuming you are using a single redis instance
Otherwise if your running two redis clusters, 1 for cache and one for queue then you would set the REDIS_CLUSTER_KEEP_ALIVE env variable

ujwal1373OP•1/19/24, 1:13 PM

@Zac Clifton I think Azure Redis has a timeout of 10 minutes by default.. that's when it is going stale. The Keep Alive variable is set to 30000ms by default which is 30 sec but still the connection is going stale. Does Novu have an ability to ping the Redis for every few minutes to keep it alive?

Uujwal1373 @Zac Clifton I think Azure Redis has a timeout of 10 minutes by default.. that's...

Zac Clifton•1/19/24, 1:51 PM

We do not have a cron job to do this if that is what you are asking, we are currently relying on the keepalive feature at this time

ujwal1373OP•1/19/24, 3:20 PM

does keepalive make sure the connection is not getting closed?

Novu_Bot•1/19/24, 3:20 PM

@Ujwal, you just advanced to level 4!

ujwal1373OP•1/19/24, 3:23 PM

do we need to explicitly set it in the env vars since the default is 30 secs anyway?

Uujwal1373 do we need to explicitly set it in the env vars since the default is 30 secs any...

Zac Clifton•1/19/24, 5:12 PM

I would recommended tring this, we have not had this problem on our side yet as we deploy in aws but I am happy to continueing helping as much as I can on the side.

ujwal1373OP•1/19/24, 5:13 PM

sure I will try it and let you know, thanks Zac!

Uujwal1373 sure I will try it and let you know, thanks Zac!

Zac Clifton•1/19/24, 6:48 PM

My pleasure

Redis connection is getting stale on idle

Similar Threads

Similar Threads

Similar Threads