A40 availability
There are a couple of >1 month old posts about this but it seems to be an issue again, A40s have become pretty much entirely unavailable other than at weird times (~7am GMT) and it's been like this for about a week now, what's going on? Availability seems unusually poor, I've never known it like this, I've got quite a lot of credit that I can't use.
32 Replies
Yeah I’ve found the same!
7 am GMT is when I, personally, usually start On-Demand these cards, as it's worktime for us here. Our team finds these cards very cost-efficient. Depending on a scale of what you're doing, it might seem that we hoard them. But this stuff is FCFS, so if you really really really need these specific cards - you can use savings plans.
I used to start and stop them based on workload, I'm not running a 24/7 workload so savings plans don't really work for me, if I could I'd get my credit out of RunPod so I could put it into an alternative provider with slightly higher pricing but much better availability but I can't
It's not the specific cards, but everything vaguely suitable that has disappeared, only the smallest cards seem to be routinely available now
I assume that L40/L40s are not desirable? We sometimes have to resort to using those, though it's twice as expensive.
L40 is a good shout, but it's not routinely available in the two regions I use (EU-SE and CA-MTL), A40s have been a bit better supplied over the past 24 hours which is nice, and I've signed up with an alternative provider which offers a better price than RunPod (~$1.15/hr) on A100 80GB which also works for me, it's good to have a backup! I'm not overly price-sensitive but don't want to spend multiple $/hr on very high-end cards just for inference.
Unknown User•12mo ago
Message Not Public
Sign In & Join Server To View
I wonder if they already have, as it seems to have improved a lot in the past few days especially in CA-MTL, either they've added more supply or a heavy user has stopped what they're doing
Unknown User•12mo ago
Message Not Public
Sign In & Join Server To View
I see we're back where we were with the A40s, I guess another week without them :/
Same for me, my serverless endpoints configured for A40 are stuck since yesterday
Unknown User•12mo ago
Message Not Public
Sign In & Join Server To View
Makes it quite difficult to justify a top up when my current credit runs out tbh
And again, I'm paying (admittedly not much) to store stuff in two locations and can't use it
Unknown User•12mo ago
Message Not Public
Sign In & Join Server To View
They're getting rid of them I think, one I managed to deploy had a "this server is being removed" maintenance message 😦
Unknown User•12mo ago
Message Not Public
Sign In & Join Server To View
CA-MTL-1
Unknown User•12mo ago
Message Not Public
Sign In & Join Server To View
Yeah
Unknown User•12mo ago
Message Not Public
Sign In & Join Server To View
My data is on network storage, so data is fine but it looks like they might be lowering the stock of A40s to make room for something else
Unknown User•12mo ago
Message Not Public
Sign In & Join Server To View
It was something like "this server is being removed to increase capacity" so sounds like they want to increase capacity of some other GPU
Unknown User•12mo ago
Message Not Public
Sign In & Join Server To View
Maybe it's already happened in EU-SE-1 as A40s seem to have been low there for a while
If they can send me a couple of the A40s they take out that'd be great 😄
Unknown User•12mo ago
Message Not Public
Sign In & Join Server To View
I'm joking, I don't have 5k to blow 😄
Unknown User•12mo ago
Message Not Public
Sign In & Join Server To View
that's about what they are on eBay!
Unknown User•12mo ago
Message Not Public
Sign In & Join Server To View
This is not quite as bad as it was previously, but still going on, normally they all disappear evening European time (which is generally when I want to use it)
This is getting worse and worse again - there's a bit of availability in the morning European time, but by midday they're just gone, across all regions, every day, it seems to be when the US wakes up
Unknown User•11mo ago
Message Not Public
Sign In & Join Server To View