Runpod

R

Runpod

We're a community of enthusiasts, engineers, and enterprises, all sharing insights on AI, Machine Learning and GPUs!

Join

⚡｜serverless

⛅｜pods

🔧｜api-opensource

📡｜instant-clusters

🗂｜hub

r1
r1
View on Discord
12/19/2023

when will the status endpoint for a serverless function return 429s?

(see title)
antoniog
antoniog
View on Discord
12/19/2023

Issue with worker-vllm and multiple workers

I'm using the previous version of the worker-vllm (https://github.com/runpod-workers/worker-vllm/tree/4f792062aaea02c526ee906979925b447811ef48). There is an issue when more than 1 workers are running. Since vLLM has internal queue, all the requests are being immediately passed to the one worker. The second worker doesn't receive any requests. It it possible to solve it? I've tried a new version of the worker-vllm but there are some other issues. Thanks!
Peps
Peps
View on Discord
12/18/2023

Throttled

Hey 🙂 Is there something I can do to prevent getting throttled? I see the availability for the GPU I selected is high, and I'm also not using any network disk, so I'm a bit confused what's exactly happening. ID: ofjdhe4djh1k5t...
Najeeb
Najeeb
View on Discord
12/18/2023

Cuda Version in serverless

Which cuda version is being used in A100 80 gb gpu at serverless ?
jmkweb
jmkweb
View on Discord
12/18/2023

4minute Serverless (Server Not Ready) Constantly.

Sometimes getting a serverless worker boot up, run for ~5mins with "2023-12-18T16:18:12.087432983Z {"requestId": null, "message": "Service not ready yet. Retrying...", "level": "INFO"}" constantly repeating. Then does nothing and deletes.
reavatar
reavatar
View on Discord
12/18/2023

Download Stuck - Worker VLLM

The download get stuck on some value https://github.com/runpod-workers/worker-vllm...
No description
Stone Johnson
Stone Johnson
View on Discord
12/17/2023

Can worker-vllm work with https://huggingface.co/TheBloke/Wizard-Vicuna-30B-Uncensored-GPTQ

I have customers that want to use this and I think worker-vllm is the way to go with it. Hope it can work with worker-vllm! Link is https://huggingface.co/TheBloke/Wizard-Vicuna-30B-Uncensored-GPTQ and I guess it works with text-generation-webui so why not worker-vllm
SwervyFree
SwervyFree
View on Discord
12/17/2023

Unable to access network volume data from serverless deployment

I have a network volume called models (id: 07k4ohphfw) I deployed a Pod (id: t4piur6ezzwjrn) to populate this network volume with models in the /workspace directory. I created multiple serverless endpoints to try and access the models in my network volume. I created have a serverless endpoint (id: k0xfhgd47d2xk8) for debugging purposes, which simply lists the files in /runpod-volume. The volume is attached to the serverless endpoint. However, runpod-volume is completely empty when I try to access it. ...
Punit
Punit
View on Discord
12/17/2023

Stop button missing

Can’t find Stop button in my pod ID 8uiw7twm1deb2y Only option is to terminate pod...
No description
m0nkspade
m0nkspade
View on Discord
12/16/2023

Not able to run Jupyter Lab?

Not other pods are running - no idea why I cannot run Jupyter Lab on port 8888 when it worked just fine the previous evening. #⚡｜serverless runpod
No description
aleon3
aleon3
View on Discord
12/16/2023

Running Spaces on Runpod Error

Following the Docker to Runpod Instruct I get the following error:
No description
Ardha
Ardha
View on Discord
12/16/2023

Fooocus run_anime.bat

Hi, i try fooocus community template today on my pods. But i want to try to run anime presets of fooocus. How i can do that?
No description
Previous