Whisper X
I have trialed the whisper X and it keeps failing to encode properly on the serverless instances. I dont see any other way to configure them. Has anyone else had the same problem with FFMPEG?
Serverless spins up needless workers during cold boot
I'm using llama 3.3 70b on vLLM using Serverless. When I don't have any workers and a new job comes in, Serverless starts a worker. However, even though I'm using cached models and flashboot, the worker will show as "running" but will not actually start processing any jobs for about 3-4 minutes. During that time, Serverless starts a 2nd and a 3rd worker because of Queue Delay.
However - starting a 2nd and 3rd worker won't do anything to help the situation, because they will also take 3 minutes before they can start accepting jobs. The only thing it does is charge me money for useless workers every single time that we go from 0 to 1 (or in this case 2-3) workers.
It seems to me that the Queue Delay should count from the time a worker is actually processing jobs, not just blindly the time the job is in the queue. Or, like any other autoscaling solution I've used, we should be able to set an upscale time limit that prevents additional upscales from happening within X minutes of an upscale....
build SERVERLESS
i am facing issues with fresh builds stuck in pending for hours. no code issues from my end but builds failing
Model loading time
i'm running a custom image that includes mixtral 8*7B model where model loading itself is taking 4.5 minutes to load each time a worker starts. i don't want to use an active worker given my load and also do not want to waste 4.5 minutes for model loading as that incurs me additional cost. any better alternatives for this. I already cache models and use network volume with the serverless endpoint.
GitHub builds for serverless stuck in "Pending"
i am facing issues with fresh builds stuck in pending for hours. no code issues from my end but builds failing
4/20 healthy workers - pruna ai image model from hub
Hallo,
I am really stack with that, raised limits to 20 workers (even 30 in theory), I get 4 healthy workers, so queue of 100 images is done by 4 workers at most.
model from hub: FLUX.1-dev-juiced 0.1.2...
Increase serverless quotas
We need to increase our serverless quota. What requirements are needed for this? I have already submitted the form via serverless interface, but there has been no feedback yet.
GitHub builds for serverless stuck in "Pending"
GitHub builds for serverless is not working. it remains as pending state after yesterday's server down
my account is knugori2025@gmail.com
screenshot says it spent only several minutes but I tried removing and rebuilding the endpoint several times but still don't work...
how can I upgrade vllm version in serverless worker image?
https://docs.runpod.io/serverless/endpoints/model-caching
Hi, I was trying the new Qwen3-next-80B-a3b-instruct in the example, and I selected vllm worker image to run it, but it seems that the vllm version in the image doesn't support the model yet. so I added the pip upgrade transformers and vllm in the docker start command, and the image still tries to load the model in the old version....
Serverless Comfyui not recognizing model_patches folder on network volume
Hi there, I've been trying to get my workflow to run on comfyui and I'm getting stuck with an error that it's returning to me. I am not using any custom nodes, just base level nodes that are included with Comfyui. My workflow contains the
ModelPatchLoader node that looks for models in the model_patches folder. I've added the folder and model to my network volume and have verified that they are there. But I keep getting this response from the endpoint:
{'delayTime': 618, 'error': 'Workflow validation failed:\n• Node 39 (errors): [{\'type\': \'value_not_in_list\', \'message\': \'Value not in list\', \'details\': "name: \'uso-flux1-projector-v1.safetensors\' not in []", \'extra_info\': {\'input_name\': \'name\', \'input_config\': [[], {}], \'received_value\': \'uso-flux1-projector-v1.safetensors\'}}]\n• Node 39 (dependent_outputs): [\'9\']\n• Node 39 (class_type): ModelPatchLoader', 'executionTime': 323, 'id': 'sync-445ef416-ddcf-4a2c-bba7-fd6bc9e192a3-u1', 'status': 'FAILED', 'workerId': 'aehi12zofrz99h'}
Would love if you are able to provide any insight into this for me. Thank you....401 Error
No serverless pods are working at the moment. Several of our features are down at the moment.
Can't send request to serverless:
Send request and tip:
500 Server Error: Internal Server Error for url: https://api.runpod.ai/v2/7ftwu8lpwgzwbc/run...
GOT 401 Error
I didn't change anythings. but i got 401 reponse with 401 Unauthorized | Make sure Runpod API key is set and valid.
Sending Image File to Serverless LLAMA3.2 Vision
Hy anyone know how to send Image File to Serverless Runpod which is using LLama 3.2 Vision becasue im struggling alot with that.
import requests
headers = {...
Serveless prices and technical questions
Hi I'm having a great time using the runpod at first it was a little bit hard to make my ai app compatible to serverless but it now working great.
I wanted to make sure what prices are.
...
Solution:
read the docs about model cache / model store its compatible with hf models
or if you're not interested with that, put the models in your network storage / inside the docker image then load it on runtime (on your handler.py, before rp.serverless.start() call)...
Message Not Public
Sign In & Join Server To View
value not in list on serverless
i have a network storage setup with comfyui that i use to deploy pods on, now i want to use that storage with serverless. i followed the guide to https://github.com/runpod-workers/worker-comfyui/blob/main/docs/customization.md , tried method 2 and created an endpoint with the runpod/comfyui-worker:5.5.0-base and set the network storage to this endpoint. when trying to use a simple workflow (flux1-dev) on serverless that perfectly works when connected with a pod, i get a error "value not in list...
Solution:
For the records: In my network storage the models from the comfyui setup to run a pod are saved in /(workspace)/ComfyUI/models/.. but the serverless worker is looking at /(runpod-volume)/models/.. , putting models there fixed the "value not in list" error on serverless. it was a matter of not reading the docs carefully enough on my side, its mentioned in the very bottom note @ https://github.com/runpod-workers/worker-comfyui/blob/main/docs/customization.md:
"Note:
When a Network Volume is correctly attached, ComfyUI running inside the worker container will automatically detect and load models from the standard directories (/workspace/models/...) within that volume....
Finish task with error: CUDA error: no kernel image is available for execution on the device
I often get this error for any of my requests. And I am more than sure that this is a Runpod problem. How can I fix this?