Runpod•3y ago

Throttled

Hey

Is there something I can do to prevent getting throttled? I see the availability for the GPU I selected is high, and I'm also not using any network disk, so I'm a bit confused what's exactly happening.

ID: ofjdhe4djh1k5t

PepsOP•12/18/23, 11:13 PM

ended up having to manually terminate the throttled worker, and after that it automatically spun up another worker that got my requests unstuck. Not sure why I had to do this manually to get my queue unstuck

Justin Merrell•12/18/23, 11:17 PM

Increase the max number of workers you have set.

PepsOP•12/18/23, 11:19 PM

in my use case I only ever want to run one worker at the same time, as I will be running a task about every 30 seconds that's not time sensitive. I wouldn't mind if a throttled worker that's stuck for several minutes gets automatically terminated and spins up another

PPeps ended up having to manually terminate the throttled worker, and after that it au...

J.•12/18/23, 11:24 PM

I think increase the maximum amount of workers, but set the queue time under advance for like 1 minute or something

J.•12/18/23, 11:24 PM

you can set the time it takes before a worker spins up a different worker

J.•12/18/23, 11:24 PM

that way u only pay for active workers

J.•12/18/23, 11:25 PM

There just some weird behaviors with a small maximum worker

JJ.I think increase the maximum amount of workers, but set the queue time under adv...

PepsOP•12/18/23, 11:28 PM

ah yeah that sounds good. In my case I'm not really doing anything time-sensitive, so I'll probably just set it at like 5 minutes so my queue is not stuck forever.
Still would be nice if workers that are throttled for a long time are automatically terminated, but yeah this should work for now. Thanks!

PepsOP•12/18/23, 11:43 PM

bit confused what's happening here now

PepsOP•12/18/23, 11:43 PM

my queue is empty for a while now, and I see this

PepsOP•12/18/23, 11:44 PM

my idle timeout is low and active workers 0, so I don't quite understand what it's trying to do, with my queue already being empty for more than 5 minutes or so

PPeps my idle timeout is low and active workers 0, so I don't quite understand what it...

J.•12/19/23, 12:21 AM

xD yea dont worry about it

J.•12/19/23, 12:21 AM

Runpod adds an additional 3 workers in the background to help with scaling issues but u arent paying for it

J.•12/19/23, 12:22 AM

nor count against ur limit

J.•12/19/23, 12:22 AM

i also see the same thing

J.•12/19/23, 12:22 AM

i wish runpod explained whatever is going on here more but never had an issue with it

PepsOP•12/19/23, 12:48 AM

ah alright

PepsOP•12/19/23, 12:48 AM

I didn't have this before when sticking with 1 max worker, so got a bit confused when seeing this after increasing the max to 2

PPeps I didn't have this before when sticking with 1 max worker, so got a bit confused...

J.•12/19/23, 1:27 AM

Yeah - is a weird thing they do when ur greater than 1 worker max.

, wish was something documented / explained but so be it

J.•12/19/23, 1:27 AM

https://discord.com/channels/912829806415085598/1185822418053386340

J.•12/19/23, 1:27 AM

I wish for it too

PepsOP•12/19/23, 1:28 AM

Utilizing network storage if your docker image is 20+ GB

oh uh

PepsOP•12/19/23, 1:28 AM

my docker image is definitely not 19GB

PepsOP•12/19/23, 1:29 AM

it's incredibly bloated because I had to go through a lot of hoops to get GPU support working with my tensorflow for some reason. Currently still building my thing as a POC, so haven't really slimmed down the image yet

JJ.Yeah - is a weird thing they do when ur greater than 1 worker max. 🤷, wish was ...

flash-singh•12/19/23, 2:49 AM

max 1 worker we treat as development, and give you only 1 worker, most people want to ssh and debug, 2 or more are considered production workload and we add additional worker caches to help reduce throttled workers

PPeps it's incredibly bloated because I had to go through a lot of hoops to get GPU su...

J.•12/19/23, 3:03 AM

haha i think under 20 gb is actually ok

J.•12/19/23, 3:03 AM

i think once it exceeds like 30 gb / climbs around that regions gets bad

Throttled

Similar Threads

Similar Threads

Similar Threads