Runpod

We're a community of enthusiasts, engineers, and enterprises, all sharing insights on AI, Machine Learning and GPUs!

Join

Runpod

We're a community of enthusiasts, engineers, and enterprises, all sharing insights on AI, Machine Learning and GPUs!

Join

⚡｜serverless

⛅｜pods

🔧｜api-opensource

📡｜instant-clusters

🗂｜hub

Jaber

2/14/2025

Github Serverless building takes too much

hey guys when I tried to do serverless using github and Docker file , it already take 1 hour in this status "Waiting for Build" and there's no log errors, what the problem?

skeledrew

2/14/2025

Websocket Connection to Serverless Failing

Hello. I'm updating a project to use websockets (moving from batch to realtime processing), however whenever I attempt to connect, I get a "websockets.exceptions.InvalidStatus: server rejected WebSocket connection: HTTP 404" error via wss://. It changes to a "HTTP 301" which redirects to https:// and also fails due to the invalid protocol if I use ws:// instead. I construct the URL as wss://<pod_id>-<port>.proxy.runpod.net/ws and expect this to be translated to wss://localhost:<port>/ws, and the websocket server is run in a thread just before the HTTP server is run. The latter works fine as I am able to communicate with it via the regular https://api.runpod.ai/v2/<pod_id> URL. The expected port is exposed in the Docker config, as per https://docs.runpod.io/pods/configuration/expose-ports. Any ideas what the issue is?...

JohnDoe

2/12/2025

Pulling from the wrong cache when multiple Dockerfiles in same GitHub repo

I seem to be having an issue where the wrong cache is being pulled by a worker when I have specified the Dockerfile in GitHub integration. Any help would be appreciated!

Edo do di

2/12/2025

Severless confusion

hi, where does the openai compatible end point option on the severless confing? i just now using severless and i unable to connect it with my front end because they need OpenAI Compatible

xsjiang

2/12/2025

How to pass parameters to deepseek r1

I have deployed deepseek r1 on serveless and then I don't know how I am going to pass parameters to it and what is the structure of its parameters. For example how do I tell the model max_tokens, what to write for the messages parameter.

blue whale

2/11/2025

Job stuck in queue and workers are sitting idle

This has been the case very often. The jobs are stuck in the queue and workers are idle. How to improve this? There was not anything else going on with any other worker (or endpoint for that matter).

ctrlaltelite

2/11/2025

Endpoint/webhook to automatically update docker image tags?

Is there a way to tell runpod there is a new docker image to update the endpoint to without doing it manually on the portal?

Zaid Qureshi (S25)

2/11/2025

What is expected continuous delivery (CD) setup for serverless endpoints for private models?

Hello, our model artificats are stored in S3, what is the continuous delivery setup for serverless models not hosted on dockerhub? What I have seen so far: - Existing runpod workers download publicly available models and push them to dockerhub...

D Z

2/10/2025

InvokeAI to Runpod serverless

Is it possible to link up InvokeAI to a serverless instance? I'm hoping to have it installed locally then use an external GPU, but not sure what is required to set that up. Wondering if there is a tutorial or something on doing so.

Paweł Biskup

2/10/2025

Comfyui From pod to serverless

Hi I've got my comfyui setup running on pod. Now which is the fastest way to make it work in serverless? I used a network volume.

Xqua

2/10/2025

Is serverless Network Volume MASSIVE lag fixed ? Is it now usable as a model store ?

Hi a while ago I tried to model store to avoid having to manage my now 100Gb docker image. But the runpod network volume took forever to load making 15 sec request take 1mn30 or more Support said they were working on a fix, is this now usable ?...

kranas4755

2/9/2025

Serverless with network storage

Hi all, I am trying to setup a serverless worker for comfyui (currently using customized template from this https://github.com/blib-la/runpod-worker-comfy . I have a several large models which I would like not to bake into the image. I see there is an option to mount network storage to serverless worker, I tried to mount it (with the required models to run the workflow) to the serverless comfy worker, but when I send a request with the workflow I see in the worker logs that it does not see any of the models in the mounted storage....

BAS014

2/8/2025

Workers keep respawning and requests queue indefinetely

Hi there I tried asking in the "ask-ai" channel. but I need some more help. "I've just deployed a servless endpoint on 3 regions, when 1 worker gets to about 7 mins running, it goes to idle then spawns a new worker. over and over. Is this normal? its a small model and workers have been running now for a bout 35 mins? I tried a request but that just goes into a queue and doesnt get completed"...

digger18

2/8/2025

Handler output logs

Is there a way to output the handler logs so it will be displayed in the worker/pod logs? I am running a 20-60min job so I would like to have visibility to the current status of the job progress....

Stewette

2/8/2025

The default steps on the website for serverless create broken containers that I am charged for.

Hi! I'm new to all of this, but I'm running into a serious issue. I've put money on my account, click "Get Started" on autoscale with serverless, click "Text" under preset models, then click the deepseek-ai/DeepSeek-R1 model. There are no prompts to configure anything - everything is made with RunPod's premade settings....

dantetwc | OCC

2/7/2025

GraphQL Issue

Hi, I am trying to retireve worker state of my endpoint ``` query Endpoints { myself { endpoints {...

faileon

2/6/2025

You do not have permission to perform this action.

Hello everyone, trying to access my serverless function, but I just can't get it to work... ``` curl --location 'https://api.runpod.ai/v2/<endpoint-id>/runsync' \ --header 'Authorization: Bearer rpa_VPG4....' ...

LisT_99

2/6/2025

vLLM serverless output cutoff

I deployed a serverless vLLM using deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B But when i made a request, output is only 16 tokens (tested many times), I don't change anything from default setting but max_model_length to 32768. How can i fix that? or did I miss any config?...

digger18

2/6/2025

"worker exited with exit code 1" in my serverless workloads

No other information in the logs. Itis a GPU (cuda) enabled container. What is the best way to debug this?...

tzushi

2/5/2025

"Error decoding stream response" on Completed OpenAI compatible stream requests

Context I have a custom worker on serverless, I am streaming a response from async OpenAI python client. Error When making requests on the OpenAI compatible API endpoint, non-streaming is fine, but stream requests always return with:...

example_request.json

Previous Next

Gaming

Programming

Runpod

We're a community of enthusiasts, engineers, and enterprises, all sharing insights on AI, Machine Learning and GPUs!

Runpod

We're a community of enthusiasts, engineers, and enterprises, all sharing insights on AI, Machine Learning and GPUs!