Slowish Downloads
I'm trying to setup a pod running ComfyUI for Flux at the moment, and it's going to take 30-40 mins just to download the models with the speed it's running at.
```Downloading 1 model(s) to /workspace//storage/stable_diffusion/models/unet...
Downloading: https://huggingface.co/black-forest-labs/FLUX.1-dev/resolve/main/flux1-dev.safetensors
0K .......... .......... .......... .......... .......... 0% 10.9M 34m23s...
can't cloud sync with Backblaze B2
I need help, I can't do cloud sync with Backblaze B2
I put the key ID and the application key and the bucket root path
but it says Something went wrong!...
How do i deploy a Worker with a Pod?
I have deployed a worker with a Serverless deployment, now i expected to be able to deploy the exact same image to a Pod and be able to have an endpoint URL to make a similar Worker request, but i'm not having success?
I am currently using the following as the initial entrypoint for handler.py...
Is there any doc that discusses how to get a Serverless Worker deployed to a Pod?
thx....
runpod.serverless.start({"handler": handler})
runpod.serverless.start({"handler": handler})
Funds not appearing in account balance
Hi - I deposited 300 dollars in my account. I got emailed the receipt. But the funds haven't been deposited as credit - could you look into this please?
Very inconsistent performance
I recently started using Runpod - and am a fan of the setup simplicity and pricing. I have recently noticed a huge amount of inconsistency in performance with identical training runs taking up to 3x longer to finish. I am on the secure cloud. Do you know why this may be?
Can someone help me fix my tensorflow installation to see the gpu?
I've been trying to fix this for over a week.
Running the official template with pytorch 2.1.0, cuda 118...
save state of pod to persistent storage?
HI,
once I'm done training with a pod, is there a way to save my storage/current state off to a 'longer term' storage so I don't have to go through setting everything up again via ssh when i do my next training session?...
There's inconsistency in performance ( POD )
Hello. I rent and operate 20 RTX4090 GPUs all day long.
However, there are significant differences in inference speeds.
Each line in the table in the attached image represents 2 RTX 4090 GPUs.
One processes 150 images in 3 minutes. However, the rest only process 50-80 images. On my own RTX4090 2-way server that I purchased directly, the throughput is 180 images processed in 3 minutes. I haven't been able to figure out why these speed differences are occurring.
The inference task is generating one image....
Pod's connection is less stable that the tower of babel
I'm trying to use ollama in a container on runpod as a pod and I keep running into connection errors over and over again. I've tried different pods, Secure Cloud vs Community and different GPUs but I keep getting timeouts like this:
...
ResponseError: <!DOCTYPE html> <!--[if lt IE 7]> <html class="no-js ie6 oldie" lang="en-US"> <![endif]--> <!--[if IE 7]> <html class="no-js ie7 oldie" lang="en-US"> <![endif]--> <!--[if IE 8]> <html class="no-js ie8 oldie" lang="en-US"> <![endif]--> <!--[if gt IE 8]><!--> <html class="no-js" lang="en-US"> <!--<![endif]--> <head> <title>3k27lkqzwstw36-11434.proxy.runpod.net | 524: A timeout occurred</title>
ResponseError: <!DOCTYPE html> <!--[if lt IE 7]> <html class="no-js ie6 oldie" lang="en-US"> <![endif]--> <!--[if IE 7]> <html class="no-js ie7 oldie" lang="en-US"> <![endif]--> <!--[if IE 8]> <html class="no-js ie8 oldie" lang="en-US"> <![endif]--> <!--[if gt IE 8]><!--> <html class="no-js" lang="en-US"> <!--<![endif]--> <head> <title>3k27lkqzwstw36-11434.proxy.runpod.net | 524: A timeout occurred</title>
Two pods disappeared from my account
After a two week hiatus from RunPod I returned frustrated to find that at least two (on-demand, secure cloud) pods are missing from my account. These take time, effort, and money to setup, and I was happily paying their storage costs. My account is set up to autopay and roll over, so the balance is always > $100 (i.e., this is not a non-payment issue). There have been no reported storage outages AFAIK, and my audit logs show no activity whatsoever between 8/17 when I last used RunPod and today. Billing however indicates one being dropped on the 23rd, and another on the 24th.
Can anyone shed some light on what's going on here, and ideally help me restore my missing pods?
Similar issues, for reference:...
Why cant I find any comfyui templates on explore!!!!
I keep searching for templates on explore and I cant find any comfyui templates.
comfyui api 401 unauthorized
I installed comfyui to runpod, which works fine in the browser.
However, when I try to access the same URL through axios to use the comfyui api, I get 401 Unauthorized. what may be the problem?...
Spot
The pricing of Spot is really too tempting. As a student with little money, this price seems very cost-effective, but it is taken up too quickly. What are you guys doing with Spot? Sometimes this thing It can only be used for 5 minutes.
Should i able to launch a pod using nvidia/cuda docker images?
I am trying to start a pod using nvidia/cuda:12.6.0-cudnn-runtime-ubuntu24.04 (to get both cuda and cudnn).
I'm not an a docker expert, but should that work?
The pod appears to start, but the licensing message keeps looping in the logs, and i can't SSH into the pod?
Any ideas? thx....
Solution:
No, you need to setup it yourself, and add a sleep infinity or a program that's running in the main thread, to make sure the container isn't looping start/stop
Message Not Public
Sign In & Join Server To View
Connecting to Pod- Web Terminal Not Starting
Good Day,
I am using my first pod and I have virtually no Linux skills. I created a pod and it's running fine. When I first created it, I connected by the "Start Web Terminal" and then clicked Connect and everything worked fine for about 30 minutes. Then it said I was disconnected. Tried to start web terminal again and it doesn't start and I cannot connect.
Tried connecting via ssh on my windows box, bit it's asking me for the root password and I have no idea how to determine what that is....
Am I downloading directly from HuggingFace when I download models?
When I download a model from huggingface, am I using up their bandwidth or does runpod have some cache server that sits between my runpod and huggingface? I feel bad downloading from huggingface, bandwidth isn't free for them and all that.
Not 1:1 port mappings for multinode training
Hi I am trying to run multinode distributed training over multiple machines, but it isn't working. I think this because when I use the torchrun.distributed command and specify the port, if I choose the internal port the other machine send data to the wrong external port. If I choose the external port, my master node doesn't listen on the correct internal port.
Is this a problem other people have had and is there a solution?
Thanks in advance!...
How to override ollama/ollama image to run a model at startup
Hi, I´m trying to run pods using the ollama template (ollama/ollama) and trying to override the default template to during pod creating serving the model that I want.
I tried to use ./bin/ollama serve && ollama run llama3.1:8b command into "container start command" but it doesn´t work. Any way to do this? Thanks!...
How to send request to a pod ?
Hello, for various reasons, I have a docker image that I want to run on a pod with a flask api. Except that I can't send any request, locally everything works but as soon as I put it on a pod it's problematic. First of all, I can't get a public IP address, so I thought I'd go through https://{pod_id}-{internal_port}.proxy.runpod.net/ping, but still, my requests don't work. So I tried using nginx to redirect requests to my container's internal port, but again, I'm having a bit of trouble with nginx, it doesn't seem to work.
Is there something I've misunderstood about how to use runpod?...
Stoped Pod Price
When we use runpod.stop_pod(pod['id']) to stop a pod, and the pod's status becomes "stopped", how is the pod billed in this state? Is the GPU resource fee still charged?