Serverless endpoint long waits in "Initializing" state
Requests to a serverless endpoint at
/run have an "Initializing" status in the dashboard for up to 15 minutes. Is this a normal queue time for an endpoint with no other requests?Foooocus too slow on generation
SD is very complexed, so i decided to use fooocus UI instead which im used to. The problem here is that the generation process takes LONG time to occur while on sd no more than 5 seconds per picture. is there a way to make fooocus generate on the same speed?
Image Generation problem
NansException: A tensor with all NaNs was produced in Unet. This could be either because there's not enough precision to represent the picture, or because your video card does not support half type. Try setting the "Upcast cross attention layer to float32" option in Settings > Stable Diffusion or using the --no-half commandline argument to fix this. Use --disable-nan-check commandline argument to disable this check. why this happens when i try to generate a pic?
could not start a temporarily closed pod
I had a pod which stopped yesterday, but now I could not start it.
Outdated controlnet how to update?
how to update controlnet to its latest version? i cant find it on updates and when i use the link from the official page it says its installed but on the ui it still shows the old interface without the new options,
Solution:
```
cd /workspace/stable-diffusion-webui/extensions/sd-webui-controlnet
git pull
source /workspace/venv/bin/activate
pip3 install -r requirements.txt...
There are no available GPUs on this host machine
why this happened? I need to secure a GPU wihtout happing such issues, i terminated and deleted the old pod and when i started new one the same thing occured. I havent managed to even use SD all this time as i keep getting such errors, how to solve this?
copy folders from one location to another, inside Jupyterlab?
Hi, is there a way to copy folders from one location to another, inside Jupyterlab? I want to install something but it installs in the wrong venv folder.
Solution:
You can't copy things in a venv from one folder to another, the venv path is hard-coded so you need to use terminal and
pip install.a6000 is apparently all gone but still available on page
trying to use a6000 on community cloud but its apparently all used, which i doubt because it still shows the ability to grab one on the main gpu select page, for now i grabbed 2x3090 and am hoping that it will work for what im trying to do
Solution:
Community cloud is supplied by community members (don't worry, they are vetted). Secure cloud is supplied by RunPod themselves. So it is possible for Secure (default) to have a GPU in supply but for community to not and vice versa
Empty trash?
I deleted files using gui and in doesn't register so I am out of space.. I really need to find a way to get some storage back and I dint want to restart the server.. Is there a way to "empty trash"?
Versioning serverless endpoints
I have tagged images on dockerhub that I am using for serverless endpoints. Main branch is tagged as v1. In order for the serverless endpoints to update with the new code, do i need to retag each code change in dockerhub as well as specify a new release in the Runpod dashboard?
Does the current system not pull down the image at the specified tag if the code has changed in the image?...
Solution:
There is an edit endpoint menu where u can say new release, pick a tag that is different and will redownload it. Yes new changes in dockerhub need to be retagged with something new
No, it doesnt auto download....
how can I find my pod's ip address?
how can I find the public IP address for my running pod? I've opened some TCP ports, but don't know what address to use to reach them.
Thanks...
"This server has recently suffered a network outage and may have spotty network connectivity." and
getting a "This server has recently suffered a network outage and may have spotty network connectivity. We aim to restore connectivity soon, but you may have connection issues until it is resolved. You will not be charged during any network downtime." that has persisted for a couple of hours now. price seems to still be ticking. is it dead? how long should I wait? ID: 1tncsp3jl5lsc5
Multinode training Runpod ports
I'm trying training a distributed models using multinode, 2xPods x8GPU 4090 for each. We cant train using torchrun, because i need the same TCP port, for each machine, so, runpod assigned me a random external port ,
command example:
NODE A:...
Feature Request / Is it possible RunpodCTL
Just sharing a wish / pending thought as a backlog wish ~
Is it possible to add a CLI command to runpodctl, where it generates SSH keys / stuff, and I can send "the public key" to another pod and stuff, and it automatically adds it to the authorized public keys etc.
And then it does a connection and a direct SCP file transfer? ...
How to mount persistent storage volume in pod?
I've created persistent storage and launched a pod from the storage UI. When I log in via ssh I can't see the storage volume. How do I find/mount it for use?
RunPod SD InvokeAI v3.3.0 Errors
When I try to run a runpod with invokeai, I just get a Server Error and Runtime Error when I try to generate an image.
ENDPOINT IS
#⛅｜gpu-cloud Hi to you all, can somebody please tell where to find the "endpoint" code ? I would like to connect to my GPU Cloud based by using Python !
It would be grand if somebody can post here an example of a working Python code to connect and use the GPU !
Thanks a lot to all those that would like help 😆...
connect ssh vscode to runpod gpu server
following the blog - https://blog.runpod.io/how-to-connect-vscode-to-runpod/
1. I have created ssh key, and added public key to account
2. I have created pod with TCL port 22
3. accessing the pod via terminal, using and running following command to turn on the ssh connection
bash -c "apt update;apt install -y wget;DEBIAN_FRONTEND=noninteractive apt-get install openssh-server -y;mkdir -p ~/.ssh;cd $_;chmod 700 ~/.ssh;echo YOUR_PUBLIC_KEY > authorized_keys;chmod 700 authorized_keys;service ssh start;sleep infinity"...Solution:
Hey got it resolved, thanks for help
I have wsl on my windows,
I was creating a ssh key on wsl and ...
environment variable not accessible from true ssh ?
I see it when using fake ssh but not using the true ssh. I am not sure how to setup this.
Pod disappeared after yesterdays maintenance
Hello, yesterday I wanted to start my Pod but I got the message that the system was down for maintenance till my (local) midnight. That's fine, but this morning I wanted to try again and my entire pod is gone. I hope its possible to recover this because quite some time making it went into it.
Checked if it was stale, it was getting close but still had 2 days on that timer.
Also plenty of credits. Bit strange how it can just disappear... Guess I can try to recreate it but a lot of work went into that one. Hope it can be restored somehow....