RunPod2mo ago

A way to know if worker is persistent ("active") or not

It would be a great help if there was some way from inside the code (e.g. environment variable) to know if the model is running on a persistent worker or not. Example use-case - If the worker is persistent, I can compile the model, it takes ~20 minutes but it is worth it so my users can get an almost 50% latency reduction. However, you can understand it is impossible to do for an ephemeral worker as it will take too long to initialize. Is there any way to do this?
1 Reply
nerdylive2mo ago
active workers resolve latency issues imo, theres still no way of getting if the worker is keeping the models in a specific time yet but feel free to put a feature request in #🧐|feedback