Giorgi Andriadze
Pod loses connection/reconnects periodically
Hey everyone, what I've noticed is that secure cloud pod drops connection and after couple of minutes reconnects.
Not sure what causes that, could it be related to server load? It's handling a lot of requests but not sure why that would cause server to disconnect.
Vllm logs don't display anything unusual. It feels like proxy is failing and starting again.
7 replies