A way to know if worker is persistent ("active") or not
It would be a great help if there was some way from inside the code (e.g. environment variable) to know if the model is running on a persistent worker or not.
Example use-case - If the worker is persistent, I can compile the model, it takes ~20 minutes but it is worth it so my users can get an almost 50% latency reduction. However, you can understand it is impossible to do for an ephemeral worker as it will take too long to initialize.
Is there any way to do this?
1 Reply
active workers resolve latency issues imo, theres still no way of getting if the worker is keeping the models in a specific time yet
but feel free to put a feature request in #🧐｜feedback