Issue with Worker Initiation Error Leading to Persistent "IN_PROGRESS" Job Status
Hi All,
While testing the endpoint, I observed that when initiating a job with an empty input in the request body (using {"input": {}} or an empty JSON {}), the worker fails to start successfully. Despite this, the job status continually displays as "IN_PROGRESS".
This problem persists without reaching a resolution or a timeout state, necessitating manual intervention to cancel the job. The log output indicates an error message: "Job has missing field(s): input." However, the job status doesn't reflect this error and remains indefinitely in progress....
Use private image from Google Cloud Artifact Registry
I'm trying to setup the authentication to GCP Artifact Registry, but without much success.
I've followed the instructions here: https://cloud.google.com/artifact-registry/docs/docker/authentication#json-key
I try to configure the credentials in RunPod using the contents of the base64 key file, but I immediately get an error from RunPod as soon as I hit save. A red popup on the top right saying "❌ Failed to Update Registry Credential".
I guess this is because the key is too long? (3185 chars)...
Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0!
I've been utilizing RunPod endpoints for the past month or so, no issues everythings been working wonderfully. Past week, a handful of my jobs have been failing. I'm not entirely sure why. I have not made any code changes and not changed the docker image that is from my template. I do notice that it seems to be waiting for GPU's to be available. but not sure why, when it finds them this error throws.
Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument index in method wrapper_CUDA__index_select...
Warming up [Billing]
Hello everyone,
I'm curious about the runpod billing on serverless endpoints. Specifically, does runpod charge for elapsed seconds when executing tasks outside the handler? In my use case, I'm concerned about potential charges during tasks such as 'warming up models.' which are not triggered via api, Can anyone provide insights or details on this matter?
Thank you!...
Solution:
Yep, you are basically charged from the moment the container starts until the moment it stops, regardless of what it is actually doing or whether the handler has started or not.
Worker not consuming jobs
I am following this tutorial: https://blog.runpod.io/serverless-create-a-basic-api/
I have submitted a few requests to the API, and the UI shows a worker is running (and I am being billed for the worker) but it doesn't seem to be consuming any jobs. All the jobs remain in the "queued" state indefinitely. There are no logs in the worker logs tab. See attached screenshots.
Requested 24gb worker (3090)
Any idea what is going on / tips on how to debug further?...
Should I be getting billed during initialization?
Trying to understand exactly how serverless billing works with respect to workers initialising. From the GUI, behaviour is inconsistent and I can't find an explanation in the docs.
I have an example where workers are pulling a docker image, one of the workers says they're ready despite still pulling the Image while the other two are in the initialising state. The indicator in the bottom right shows the per second pricing for one worker which would make sense if its active, but it clearly isn't ready to accept jobs.
Also, pulling images from Github container registry takes an absolute age, I'd be disappointed about getting charged more because of network congestion....
Solution:
we have seen this happen if you update your container using same tag
[RUNPOD] Minimize Worker Load Time (Serverless)
Hey fellow developers,
I'm currently facing a challenge with worker load time in my setup. I'm using a network volume for models, which is working well. However, I'm struggling with Dockerfile re-installing Python dependencies, taking around 70 seconds.
API request handling is smooth, clocking in at 15 seconds, but if the worker goes inactive, the 70-second wait for the next request is a bottleneck. Any suggestions on optimizing this process? Can I use a network volume for Python dependencies like I do for models, or are there any creative solutions out there? Sadly, no budget for an active worker....
Solution:
Initializing models over a network volume can inherently be slow bc ur booting from a different harddrive. If u can is easier to bake into the docker image as ashelyk said.
Ur other option is increase idle times after a worker is active that way ur first request is initialized the model into vram and subsequent requests are easy to pick up for the worker...
