Runpod

We're a community of enthusiasts, engineers, and enterprises, all sharing insights on AI, Machine Learning and GPUs!

Join

Runpod

We're a community of enthusiasts, engineers, and enterprises, all sharing insights on AI, Machine Learning and GPUs!

Join

⚡｜serverless

⛅｜pods

🔧｜api-opensource

📡｜instant-clusters

🗂｜hub

jdon

4/18/2024

connect dockerhub with runpod

I created a docker image locally and pushed it to dockerhub. I am now trying to make a new template in runpod using that image. But how do i fetch the image from dockerhub?

Volko

4/17/2024

Why is my endpoint running ? I don't have any questions and the time idle is set to 1 sec

Solution:

Okay so the reason why is because I enabled Active Workers

Morris

4/17/2024

idle time duration

Hi, i am running a comfyui script on serverless endpoint. I have set the idle timeout to 5 seconds but the first worker keeps running up to 7 minutes after the last request got executed. Could you explain why it's keeping running for such a long time? (ywzdlqxw0yg6ec)

briefPeach

4/17/2024

can't deploy new workers when I haven't reached limit

as you can see in the screenshot, I only have 1/5 total workers deployed. but I'm seeing this "Upgrade Total Workers" dialog that preventing me from add new workers. I previously has reached the limit of 5, but then I deleted lots of workers, so I'm now down to 1 and hoping to add new workers. But cannot add....

Solution:

oh it suddenly recovered now...

Bogossian

4/17/2024

faster whisper serverless took too much time for an small audio of 10s

Hello, i have an issue with faster-whisper, it takes too much time , gives me "IN PROGRESS" after 2 minutes, for an audio of 10seconde , it works fine maybe 1 time out of 10, have you the same issue ?

 {"delayTime":88934,"id":"sync-8a923a3f-d299-4117-b1c3-48dabdcc4bfb-e1","retries":1,"status":"IN_PROGRESS"}

 {"delayTime":88934,"id":"sync-8a923a3f-d299-4117-b1c3-48dabdcc4bfb-e1","retries":1,"status":"IN_PROGRESS"}

...

partie1.mp3

Mateusz

4/17/2024

Running serverless endpoint locally

Hi, I'm trying to run locally, but can't find a solution in documentation. I tried running handler with docker and outside docker. running with --rp_serve_api ERROR: [Errno 99] error while attempting to bind on address ('::1', 8000, 0, 0): cannot assign requested address ...

Solution:

python3 -u rp_handler.py --rp_serve_api --rp_api_port 8010 --rp_api_host 0.0.0.0

python3 -u rp_handler.py --rp_serve_api --rp_api_port 8010 --rp_api_host 0.0.0.0

...

pazanchick

4/17/2024

'Connection reset by peer' after job finishes.

Previous logs indicate that the handler works correctly. Happened multiple times now and returns failure response. Any input on this issue? @Papa Madiator ```py 2024-04-17T07:48:57.719183260Z {"requestId": "92b3176b-81d0-4dbb-9307-9cbe812dd8f0-u1", "message": "Finished.", "level": "INFO"} 2024-04-17T07:49:03.090390819Z {"requestId": null, "message": "Failed to get job. | Error Type: ClientOSError | Error Message: [Errno 104] Connection reset by peer", "level": "ERROR"}...

ribbit

4/17/2024

Connection reset by peer

Hi, I am encountering this error while pulling new docker image.

failed to pull image: read tcp 192.168.23.16:59484->104.16.98.215:443: read: connection reset by peer

failed to pull image: read tcp 192.168.23.16:59484->104.16.98.215:443: read: connection reset by peer

I initially encounter the error while running my endpoint, it'd randomly got stuck in 'in queue' status with the same error connection reset by peer. So I tried downgrading my docker image version but it failed on pull. Is there any way to fix this? Thanks...

Solution:

now it's stable, i think they fixed it

k916

4/17/2024

How to use Loras in SDXL serverless?

I don't see any docs regarding adding Loras in the workers for SDXL. I am assuming this is the worker that I should be using. https://github.com/runpod-workers/worker-sdxl...

Solution:

Yeah, the runpod SDXL worker doesn't support LoRA

yulei

4/16/2024

Tutorial about Serverless

It's anything about Serverless tutorial? Only one post on runpod blog. Can I find more?

leduyson2603

4/16/2024

Runpod return {'error': 'request does not exist'}

I'm using stable-diffusion-forge template. I have sent requests to runpod using /run endpoint and /status endpoint for retrieving the status. Everything work normally until today when im checking the error log, it shows {'error': 'request does not exist'} when sending the /status. I think the error above is from runpod webserver. Can anyone explain about this?

Solution:

If you use /runsync your request status expires in 60 seconds, and if you use /run, it expires in 30 minutes, so you need to retrieve the status before it expires. I also think you're confusing pods with serverless, because you don't use a stable-diffusion-forge template in severless, you are probably using some custom one.

Mateusz

4/15/2024

Modify a Serverless Template

Hi, I'm trying to automate docker image deployments. In documentation there is a method to modify serverless template (saveTemplate), but it's weird that I can't for example exclude env from curl request, it throws an error "GRAPHQL_VALIDATION_FAILED". Is there a way to just modify docker image name and not to touch any other field? Is there a way to restart endpoints that are using this template? I cannot find definition to get all endpoints or templates. ...

ggali3n

4/14/2024

Downloads from output

What is the best way to download files from the output folder?

rafael21@

4/12/2024

AWS S3

Guys, does anyone knows how to find these things on AWS S3? I need to add them to the runpod template. BUCKET_ENDPOINT_URL= # S3 bucket endpoint url BUCKET_ACCESS_KEY_ID= # S3 bucket access key id BUCKET_SECRET_ACCESS_KEY= # S3 bucket secret access key...

Wilkens Gomes

4/12/2024

Convert from cog to worker

I have a docker image created with the cog template and it works locally. I would like to convert it to run on serverless. I managed to convert by uploading to replicate and then using cog-worker but I would like to convert directly from local docker.

Solution:

in theory all you would need is to edit this part of dockerfile FROM r8.im/${COG_REPO}/${COG_MODEL}@sha256:${COG_VERSION}...

esho

4/12/2024

Faster Whisper Latency is High

I test a 10-second audio , and i get latency about 1 second on RTX4090 after cold start. The default is base model, and on my own RTX3090, the latency is about 0.2s.

leduyson2603

4/11/2024

Limited choice for network volume region

I want to ask why my region options for network volume in my account only support 3 regions?

digigoblin

4/11/2024

Problems with Network storage in CA?

It looks like there is a problem with Network storage in CA region that is dramatically impacting my slow start time. It is taking almost 2 minutes for the ComfyUI API to start, and this is not normal.

Xangelix

4/11/2024

Questions on large LLM hosting

1 I see mentions of keeping a model in a Network Volume to share between all endpoints. But if I already have my model inside of a container image-- wouldn't my model already be cached in that image? Which would be faster for cold boots? 2...

QDC

4/9/2024

Help with instant ID

Hello everyone, I am a Computer Engineering student with a basic coding understanding and very new to the AI space and need to do a project where i use something like InstantID to generate pictures from people. I think i can figure out how to get the model working using the example code but can't figure out how or where i can use custom models or LoRa's to get the style I want. Next to that i want it to be ran from an API of sorts because the final product wont have sufficient hardware. Does anyone know what service i can use for it that has clear instructions? I have looked into Replicate but there i dont seem to be able to add custom models at all so i think i need a different solution, this is where i stumbled upon runpod but i can't seem to figure out where to start....

Solution:

https://github.com/ashleykleynhans/runpod-worker-instantid

Previous Next

Gaming

Programming

Runpod

We're a community of enthusiasts, engineers, and enterprises, all sharing insights on AI, Machine Learning and GPUs!

Runpod

We're a community of enthusiasts, engineers, and enterprises, all sharing insights on AI, Machine Learning and GPUs!