16 GB GPU availability almost always low
Hence very frequent throttling workers and pulling docker image again and again
YAre you deploy to all data centers or you selected specific regions?
SAll data centres
DWhat CUDA versions do you have selected? Is it possible for you to widen your range a little bit more?
SI have allowed all CUDA versions
JAnd your max workers? set it to a larger number
Jmaybe share your endpoint id?
Smax workers varies per endpoint, 2-3-5 mostly
but the problem is if I set all enpoints max workers to a large number i'll hit my max workers limit
many endpoints running in parallel
but the problem is if I set all enpoints max workers to a large number i'll hit my max workers limit
many endpoints running in parallel
Jand like all of them got throttled?
Jlike 3 all got throttled?
Syes almost all keep getting throttled , and initializing in some time gaps
P@Solidsoldier
Escalated To Zendesk
The thread has been escalated to Zendesk!
Ticket ID: #22526
Jtry open a ticket
SKk will do
