Runpod

We're a community of enthusiasts, engineers, and enterprises, all sharing insights on AI, Machine Learning and GPUs!

Join

Runpod

We're a community of enthusiasts, engineers, and enterprises, all sharing insights on AI, Machine Learning and GPUs!

Join

⚡｜serverless

⛅｜pods

🔧｜api-opensource

📡｜instant-clusters

🗂｜hub

agentpabibi

7/21/2024

runpod IP for whitelisting for cloud storage

I have cloudinary account, and from runpod I want to download images from cloudinary but I want it also to be secure so what IP to whitelist so my cloudinary account only accepts request from my runpod serverless request.

agentpabibi

7/21/2024

how can I use javascript on worker code

how to do the handler file using javascript? is it possible? and if yes, what will be the equivalent of this(image attached) on javascript, I saw there is javascript sdk but on what I am seeing, I think its just for calling the endpoint? am i right? or I can also use runpod.serverless.start(), if yes what should I install and import?...

jrocky24

7/20/2024

Serverless Always IN_QUEUE?

Hi, I am pretty new to using Serverless in Runpod, and upon setting up my API and including my docker image with a Python script for making inferences (receiving and returning JSON), all the requests goes in and gets stuck at IN_QUEUE all the time. Testing my inference script locally does not cause any issues, is this an issue with the configuration?

pxmwxd

7/20/2024

Serverless doesn't scale

Endpoint id: cilhdgrs7rbzya I have some requests which requrie workers with 4 GTX 4090s. “max worker” of the endpoint is 150 and “Request Count” in Scale type is 1. When I sent 78 requests concurrently, only ~20% of these requests could start in 10s. P80 need to wait for ~600s. ...

Lance

7/19/2024

Unused HPC power

Hi, We have many of the following machines at a DC with unused computing power: GPU: 1x NVIDIA A100 80GB CPU: 14 vCores (Epyc Milan) RAM:120 GB ECC...

raz5020

7/19/2024

connecting a telegram bot to a serverless pod

Hey guys, would love ur assist on the following issue, i have a serverless function and i have an api endpoint from runpod, to which i can access using curl when providing an autorization in the http header, the problem is when using /setWebhook when creating a telegram bot i need to provide an endpoint and i can't pass on parameters in the http request, ...

tzk

7/19/2024

How to get worker to save multiple images to S3?

Hey all - my comfyui workflow is saving multiple images from throughout the workflow......however in the S3 upload, the worker is only saving one image - do you know how I can have it to save the multiple images into the same directory in S3?

Bill_secflow

7/19/2024

Using SSH to debug serverless endpoints

Hello! I had a quick question I was hoping someone could help with. In the RunPod documentation Overview section for Serverless, it states: Metrics and Debugging: Transparency is vital in debugging. RunPod provides access to GPU, CPU, Memory, and other metrics to help users understand their computational workloads. Full debugging capabilities for workers through logs and SSH are also available, with a web terminal for even easier access....

Solution:

Oh on serverless you just connect with the connect button when it's active

Message Not Public

Ucodia

7/19/2024

Serverless SDXL Turbo endpoint returning seed inconsistent images

I deployed a serverless SDXL Turbo endpoint but it returns different result despite using the same input seed and settings. Works fine with SDXL. Am I missing something?

cookiemonster

7/18/2024

Can we autoscale past 100 GPUs?

Reading the serverless documentation, under the autoscale section, it says "Dynamically scale workers from 0 to 100 on the Secure Cloud platform, which is highly available and distributed globally. This provides users with the computational resources exactly when needed." Not sure if 0 to 100 is meant literally or figuratively- Our current provider has around 50 H100s available so this is an active point of investigation for us. TLDR: Can we scale past 100 GPUs on enterprise plans? Is there an enterprise POC I can reach out to?...

tzk

7/18/2024

S3 uploads have stopped working - despite environment variables set up for template

My worker is not uploading to S3 - despite me setting up the environment variables for S3 in the template. The error in the log is "No bucket endpoint set, saving to disk folder 'simulated_uploaded'" - Interestingly this worked fine previously. I just created a new image and it no longer works. Can you advise on waht to do?

Solution:

@tzk you have wrong ENV names

Kushagra

7/18/2024

Lightweight docker image for inference generation.

Hello All, I am currently using pytorch/pytorch:2.2.1-cuda12.1-cudnn8-runtime image for my servreless endpoint. The issue is that my Github action to build and push the docker image fails due ERROR: Could not install packages due to an OSError: [Errno 28] No space left on device Is there any recommended lightweight docker image that I can use?

untech

7/17/2024

How to remove endpoint via Python API?

Hello, in Python API, you can create new endpoints. Is there a way to remove endpoints? Also, I can't find a way to get a list of templates programmatically. Thanks!

Solution:

or, if you just want a python function: ```python import requests import runpod ...

jamiep

7/17/2024

My serverless endpoint threw an error, the queue of jobs didn't get cleared, credit drained

My serverless endpoint threw an error, but job remained in the queue and new endpoints were started to process it. I was away from my desk for lunch and my account balance got drained. Is this the expected behaviour?...

Kushagra

7/17/2024

How to update a serverless endpoint with a new version of the docker image?

When we push the new version image of a docker image in the docker hub, does the serverless endpoint automatically update the workers, or do we need to publish the new image manually on the endpoint?

underdog__spider3099

7/17/2024

text generation inference docker image on serverless?

Hi i have created a template using tgi docker image and in docker commands i have entered --model-id <llama-3-8b> hf repo name and --port 8080 and choose 24gb gpu and ran a serverless instance. But i am not able to connect to this worker what i mean is when i try to ask a question, question is not being sent to the worker, but when i try to ssh into worker and asked a curl request curl 127.0.0.1:8080/generate_stream \ -X POST \ -d '{"inputs":"What is Deep Learning?","parameters":{"max_new_tokens":20}}' \ -H 'Content-Type: application/json' ...

ronit bakshi 2001

7/16/2024

No billing statement

Hey, I recently used runpod like two and a half months back and wanted to use it again. I had like 10 dollars in my account at the time and came back to see that it was empty now. I figured that this might be due to storage or some other network charges but I could not for the life of god find the distribution of how I was charged or anything in the the billing section for that matter. It's all empty. Can someone please tell me why this might be the case...

EngrBilal.

7/16/2024

Status "in-queue"

Hi! I am running my serverless endpoint in sync mode. Sometimes I get response like
{'delayTime': 85437, 'id': 'sync-822bbbf3-bae5-4efa-bbfa-9658ffda0175-u1', 'status': 'IN_PROGRESS'} or status in-queue. 1) why in sync mode it sends back in complete response? ...

RanusVietnam

7/16/2024

ComfyUI_InstantID/load_insight_face error

Hi guys, Please help me to solve error below when i run runpod serverless Thanks billion ❤️ ❤️ ❤️...

galakurpismo3

7/15/2024

Can't use GPU with Jax in serverless endpoint

Hi, I'm trying to run a serverless worker to perform point tracking on a video. It works ok, but I think that it is running on CPU. I read that the telemetry on the UI isn't reliable, but the Container Logs indicate that too. There is an image of what they logs say. It finds the Nvidia GPU, but there are problems with Jax I think. I use the function on the first image to check the device:...

Previous Next

Gaming

Programming

Runpod

We're a community of enthusiasts, engineers, and enterprises, all sharing insights on AI, Machine Learning and GPUs!

Runpod

We're a community of enthusiasts, engineers, and enterprises, all sharing insights on AI, Machine Learning and GPUs!