A way to know if worker is persistent ("active") or not
It would be a great help if there was some way from inside the code (e.g. environment variable) to know if the model is running on a persistent worker or not.
Example use-case - If the worker is persistent, I can compile the model, it takes ~20 minutes but it is worth it so my users can get an almost 50% latency reduction. However, you can understand it is impossible to do for an ephemeral worker as it will take too long to initialize.
Is there any way to do this?
Example use-case - If the worker is persistent, I can compile the model, it takes ~20 minutes but it is worth it so my users can get an almost 50% latency reduction. However, you can understand it is impossible to do for an ephemeral worker as it will take too long to initialize.
Is there any way to do this?
