Nvidia-smi parsing error
Hi,
I’m encountering an issue when launching a video generation on my RunPod instance.
Sometimes, this error message appears repeatedly:
error creating container: nvidia-smi: parsing output of line 3: failed to parse (pcie.link.gen.max) into int: strconv.Atoi: parsing "": invalid syntax
When this happens, it creates an infinite loop, and the workers keep running endlessly without stopping.
As a result, the instance keeps consuming resources for nothing.
Does anyone know how to fix this or what might be causing it?
Thanks in advance for your help! 🙏
2 Replies
Hey @Gauthier - DarLink.ai dont worry runpod customers have been warning them for months
And they still didn't do anything about it
Nor replied to customers mails
So.. Enjoy being part of the runpod downfall where workers are throttling, workers aren't even booting, the support team doesn't reply to customers, and you lose hundreds of euros of bugs that happen on runpod side.
Soon you'll discover that the same workflow on the same endpoint on the same gpu can be 6 times slower (from 30s to 3-4mins) because they put a non optimized cpu paired with a high end gpu card
NVIDIA cards are known to be a little washy in this regard, we are happy to have this problem fixed when it's reported and run a large support team who responds on
help@runpod.io in less than 24 hours.
I've identified this Pod and flagged the machine for a restart, just waiting on the datacenter.