Runpod

We're a community of enthusiasts, engineers, and enterprises, all sharing insights on AI, Machine Learning and GPUs!

Join

Runpod

We're a community of enthusiasts, engineers, and enterprises, all sharing insights on AI, Machine Learning and GPUs!

Join

⚡｜serverless

⛅｜pods

🔧｜api-opensource

📡｜instant-clusters

🗂｜hub

fanbing

3/25/2024

How to download image from s3?

When I use requests，a error "cannot identify image file <_io.BytesIO object at 0x7f59d0153600>" .

Augenbrauensenker

3/24/2024

Is execution timeout per request or per worker execution?

https://docs.runpod.io/serverless/endpoints/send-requests#--execution-policy "Execution Timeout: Specifies the maximum duration that a job can run before it's automatically terminated." The endpoint edit UI says - "Maximum amount of time in seconds a request can run for."...

Alikarami

3/24/2024

S3 ENV does not work as described in the Runpod Documention

Hi all, I have a serverless function and also all env variable as its written in documention. But its undefined in the runpod logs. I have added all this example of ENV to my template but none of them are working. ``` logger.info("env1", os.environ["BUCKET_ENDPOINT_URL"]) logger.info("env2", os.environ.get("BUCKET_ENDPOINT_URL"))...

Augenbrauensenker

3/24/2024

GPU type prioritization seems to have stopped working on 13th of March

I have an endpoint with 3 cheapest GPU types selected in the order of their price (i.e. 4090 is my 3rd option). I noticed that my expenses almost doubled after 13th of March, but the billed GPU time didn't change much. It turned out that before 13th of March the usage of 4090 was close to 0 (which is what I expect - it is my 3rd option). On 13th of March 4090 usage jumped to 50-70% skyrocketing my prices. This is unexpected and unfortunately leads to additional costs for me. For now I removed 40...

big.slam

3/24/2024

How to run OLLAMA on Runpod Serverless?

As the title suggests, I’m trying to find out a way to deploy the OLLAMA on Runpod as a Serverless Application. Thank you

Solution:

Ollama has a way to override where u the models get downloaded. so u essentially create a network volume on serverless under /runpod-volume is where they get mounted for serverless And when ur ollama server starts through a background script on start, u do whatever u want. overall its a bit of a pain...

rmnvc

3/23/2024

Serverless: module 'gradio.deprecation' has no attribute 'GradioDeprecationWarning

Hello! I'm getting this error when i use RunPod Fast Stable Diffusion with serverless. Can you please tell me what the problem might be? When i run the automatic1111 in POD it works fine, but when im using it in serverless, i get this. Gradio is 3.41.2...

minu

3/22/2024

Img2txt code works locally but not after deploying

I am using a model for Image 2 text , i have made its handler file and tested it locally , for testing i have used a json file with just defining the input { "input": { "image_path": "/content/1700052015451vm8aj9ac.png"
}...

rewbs

3/22/2024

Docker image using headless OpenGL (EGL, surfaceless plaform) OK locally, fails to CPU in Runpod

Hi all, I'm wondering if anyone can educate me on what would be causing this difference in behaviour when running a container locally versus in Runpod, and whether there is a solution. In summary I'm trying to run a headless OpenGL program in a docker container, by using ELG with the surfaceless platform (https://registry.khronos.org/EGL/extensions/MESA/EGL_MESA_platform_surfaceless.txt). I was able to get the program working as intended in a container outside of Runpod. But once deployed to Runpod, it falls back to CPU processing. As a minimal testcase, it's sufficient to simply run eglinfo, a utility which tells you what EGL devices are available. Outside of runpod multiple are available, but in Runpod none are. The testcase and example outputs are available here: https://github.com/rewbs/egldockertest ....

AC_pill

3/22/2024

Moving to production on Runpod: Need to check information on serverless costs

Hi team. I'm working with my company to move our product to release, with a soft launch in April. We ran tests already on serverless, but we need to confirm some information. ...

fanbing

3/20/2024

Serverless prod cannot import name "ControlNetModel"

When I 'runpodctl project dev', it run well in dev. But when I 'runpodctl project deploy'，run, it show : Traceback (most recent call last): File "/runpod-volume/a87e0d01/prod/instantid/src/handler.py", line 12, in <module> from diffusers.models import ControlNetModel...

jia_softmax

3/19/2024

would not execute a for loop to yield for whatever reason when streaming

Llama-index has a agenerator function in query that you can call such as this: result = query_eng.query(question) for whatever reason in runpod container, it wouldn't execute
for response in result.response_gen: print(f"response from query: {response}") yield {"word": response} ...

ssssteven

3/19/2024

S3 download is quite slow

Hey all, I just learnt that my workers spent 4s to download total ~5mb files from s3. Is that normal? Or is the best practice to include these files in the runpod payload? If so, is there any size limit on the runpod post request? Thanks!

Hello

3/19/2024

No module "runpod" found

Hi, I am trying to run a serverless runpod instance with a docker image. This is my dockerfile: ```dockerfile Base image -> https://github.com/runpod/containers/blob/main/official-templates/base/Dockerfile...

minu

3/19/2024

Captured handler exception

while using runpod sdxl api i usually get this in between running code, when i call the output, and it gets fixed after some time , p.s there is no chnage in code (because sometimes for the same payload it works sometimes it shows this) { "error": "'NoneType' object has no attribute 'split'" }...

Martin

3/18/2024

How to load model into memory before the first run of a pod?

In the template worker, in the handler file it is written:

# If your handler runs inference on a model, load the model here.
# You will want models to be loaded into memory before starting serverless.

# If your handler runs inference on a model, load the model here.
# You will want models to be loaded into memory before starting serverless.

...

alcastel

3/18/2024

Increase number workers

I have at the moment a number of workers of 10. How can I increase the maximum number of them ?...

ashleyk

3/18/2024

High execution time, high amount of failed jobs

36% of my serverless jobs have failed this morning and the execution time has spiked massively. This is completely UNACCEPTABLE. What is the issue in SE region? I want a refund for this unacceptable service.

abtx

3/18/2024

How do I write handler for /run

When I create a handler runpod.serverless.start({"handler": async_generator_handler}) Only http://localhost:8000/runsync triggers async_generator_handler However when posting against http://localhost:8000/run async_generator_handleris not triggered. Just returns ``` {"id":"test-20023617-4048-4f73-9511-8ae17a1ad7a5","status":"IN_PROGRESS"}...

abtx

3/18/2024

How do indicated job status in a handler?

For example in https://docs.runpod.io/serverless/workers/handlers/handler-async ``` import runpod import asyncio ...

zkreutzjanz

3/17/2024

A6000 serverless worker is failing for an unknown reason.

In the last week a few of our serverless workers have been failing on all requests. Trying to narrow down a common denominator right now, seems to just be an A6000 issue.

Previous Next

Gaming

Programming

Runpod

We're a community of enthusiasts, engineers, and enterprises, all sharing insights on AI, Machine Learning and GPUs!

Runpod

We're a community of enthusiasts, engineers, and enterprises, all sharing insights on AI, Machine Learning and GPUs!