Missing 12.9 cuda version in Create pod API
Hi, I'm trying to create a pod with APIs with vllm 0.9.2 on A40/A6000 in EU-SE1 how can I force it to be created on an host with cuda 12.9? vllm 0.9.0+ requires cuda 12.8+ but the hosts with a40 there have a mix of 12.4, 12.7 and 12.9. If I don't set a cuda version it work when I'm lucky to get an host with 12.9 otherwise it won't work.
Is it possible to get the cuda version 12.9 added to the API options to create a new pod ?
5 Replies
Hey, I have an issue filed for this that I thought we fixed :wires: - I'll work on this for you
I just dropped a PR for this, it may need a day or so to release. This service requires manual intervention and monitoring because of the traffic
@Dj Thank you. Looking forward for the release
@Dj It's been a few days. Do you have an estimated date for release ?
@Dj do you have any update ? There are very few gpu in EU with more or less the same spec/price with 12.8+ and even fewer with 12.8.
Sorry, I don't have permission to deploy services and noone's just done it yet :/ I'll make this happen now.
If its not 13.0 your already behind dj 😛
@Dj Is really needed more than a month to add a simple option that is already present in the web ui ? Am I the only one that is using apis ?