Kill a pod from the inside?
Last weekend I started a community pod for a large workload and went to bed once it confirmed it was starting the work properly. Unfortunately though the pod was on a very slow connection to my cloud storage, and so it spent about 14 out of the 16 hours run time just downloading the job files… I’ve only just realised it after noticing how much faster things went on other runs and analysing my cloud egress logs.
I’ve rewritten my code to report current download speeds so I can kill pods by hand, but is there any way to do it from a running python app? Ideally if it detected slow disk or downloads it’s kill itself so that at least I’d know.
My alternative is to have it send me a discord message, but that’s not as useful!
I’ve rewritten my code to report current download speeds so I can kill pods by hand, but is there any way to do it from a running python app? Ideally if it detected slow disk or downloads it’s kill itself so that at least I’d know.
My alternative is to have it send me a discord message, but that’s not as useful!
Solution
Thanks, can se that combined with some pre-set environment variables https://docs.runpod.io/pods/references/environment-variables the podStop command is what I'd need - I didn't realise that the pod knew who t was (so to speak)
