Runpod•2y ago•

31 replies

Deploying MIGAN model to Serverless.

Hi, could someone recommend how to deploy the MIGAN model in a RunPod serverless wrapper?

There are 2 areas where I need advice:

Downloading the model
Invoking the model from Python to expose it in the RunPod wrapper.
I already have an implementation, but it is very bad and slow, when it should be very fast, as the model is very lightweight (<50MB) and even runs well on CPU.

I'm taking this to Docker and from there, setting up the serverless environment in RunPod. What tips can you share to implement this model effectively?

The model: https://github.com/Picsart-AI-Research/MI-GAN

Thanks!

GitHub

GitHub - Picsart-AI-Research/MI-GAN: [ICCV 2023] MI-GAN: A Simple B...

[ICCV 2023] MI-GAN: A Simple Baseline for Image Inpainting on Mobile Devices - Picsart-AI-Research/MI-GAN

Jason•7/13/24, 1:53 AM

read this

Jason•7/13/24, 1:53 AM

https://github.com/Picsart-AI-Research/MI-GAN?tab=readme-ov-file#third-party-implementations

GitHub

GitHub - Picsart-AI-Research/MI-GAN: [ICCV 2023] MI-GAN: A Simple B...

[ICCV 2023] MI-GAN: A Simple Baseline for Image Inpainting on Mobile Devices - Picsart-AI-Research/MI-GAN

giustiroOP•7/13/24, 3:14 PM

Hi, thanks for your anwser @nerdylive , would you mind to explain your pov? That's how already i've deployed on serverless, i mean, this is the work flow right now:

-A job is fired.
-The serverless worker starts running.
-The container is up.
-Inside the container, the iopaint API is mounted.
-It looks for the Model weights (can't figure out how to persist them).
-No Model present so it will download it.
-There, the job params are passed to the iopaint API.
-Iopaint process the request.
-The response (in happy path) all way back until container and worker are released.

How it should be done in my opinion:

-A job is fired.
-The serverless worker starts running.
-The container is up.
-MIGAN model is ready (no download required) and it *directly *process the job's req/params.
-The response (in happy path) all way back until container and worker are released.

In other words, I want to avoid to use iopaint wraper. I don't know much about python and the iopaint 'wraper' is a bit complex to me.

If someone can help or advice I would be very gratefull!

giustiroOP•7/13/24, 3:23 PM

I can share snippets of code if needed of course

Jason•7/13/24, 3:26 PM

Ask on iopaint?

Jason•7/13/24, 3:26 PM

Im not really familiar with them so try exploring codes, docs

Jason•7/13/24, 3:27 PM

So sorry can't help this further except if you got the code that accepts some kind of input and outputs Ill try to help implementing it on runpod for this migan model

flash-singh•7/13/24, 3:49 PM

@giustiro bake the model into the container image

Jason•7/13/24, 3:55 PM

yeah or put it on network storage

giustiroOP•7/14/24, 6:12 PM

Hi. thanks for your inputs, I've deployed a new docker version with updates.

As I mentioned before, I had a working example, sometimes it was filing, but working anyways.

Now I need some help, the exact same endpoint template, with new Docker tag (updated from 0.2.4 to 0.2.7) it is throwing a lot of logs and it always fails!

There were no significant changes, only tried to place the model relative to the

app

app

src

src

folders, so the model is ready to use.

please @nerdylive @flash-singh can you take a look?

attaching the logs.

logs-envious_coral_stork_3.txt23.31KB

giustiroOP•7/14/24, 6:16 PM

Observations:
#1
HTTPConnectionPool(host='0.0.0.0', port=5000): HTTPConnectionPool(host='0.0.0.0', port=5000): its expected to be there until the internal FastAPI builds. Then the retry mechanism will hit it with job parameters. I repeat, THIS WAS WORKING around a month ago.

But there are a lot of other errors I haven't seen before, even the 0.0.0.0:5000/v1/api/inapint fails over and over.

#2
When serving the docker locally (with extra arg --rp_serve_api, everything starts as expected and no strange logs are shown.

giustiroOP•7/14/24, 6:19 PM

#3
-Docker base image: runpod/base:0.6.2-cpu
-python sdk: runpod==1.4.2

giustiroOP•7/14/24, 8:10 PM

here it is, the exact same container, running perfectly on my end.

Jason•7/14/24, 10:23 PM

Hey

Jason•7/14/24, 10:23 PM

What kind of errors are you experiencing?

Jason•7/14/24, 10:23 PM

Are you maybe seeing pods as serverless? Also which one are you using?

Jason•7/14/24, 10:28 PM

I believe its serverless, please help me with posting the error you're experiencing ( full line )

giustiroOP•7/15/24, 2:29 AM

Hi, thanks for your time, sure, the logs is plenty of errors and tracebacks, please do check the logs i've shared, let my attach it once again. Please open it and you will clearly see what I'm telling you.

In the other hand, when i ran the container locally, there is nothing but the expected messages from pythong indicating FastApi wrapers are up and running.

logs-envious_coral_stork_3.txt23.31KB

giustiroOP•7/15/24, 2:31 AM

And this is the serverless config

Jason•7/15/24, 2:38 AM

Hows your handler file

Jason•7/15/24, 2:38 AM

ND dockerfile?

giustiroOP•7/15/24, 2:42 AM

I have good news, it looks like the amount of logs showed can be dismissed, because I've tried my original config and it worked fine in the runpod' serverless.

giustiroOP•7/15/24, 2:42 AM

So let me tweek some configs and will comment back my updates

Jason•7/15/24, 3:02 AM

Alright great

giustiroOP•7/15/24, 3:43 AM

Well, everything is running fine now and the model is present on the container and accessible to the FastAPI. This is something!

Next improvements could be: Dismiss iopaint wrapper and understand how to run Migan jobs directly in a more performant and easy way.

Thanks for all your feedback.

Jason•7/15/24, 4:06 AM

Fast api?

Jason•7/15/24, 4:07 AM

Isn't serverless has it own built in api endpoints

Jason•7/15/24, 4:07 AM

You're using that right?

giustiroOP•7/15/24, 12:59 PM

Hi, I know this is weird (it was one of the points I wanted to ride off), but YES. Inside the container, in serverless endpoint, I have no option (because I dont know how to build an alternative, that was what I was asking help for), to use the iopaint wrapper to run migan model, and this, is served upon a FastAPI instance.

https://github.com/Sanster/IOPaint/discussions/420

Sorry if my comments are unclear, ive tried to explain the best and english is not my main language.

GitHub

IOPaint Beta release · Sanster IOPaint · Discussion #420

IOPaint Beta Release It's been a while since the last release of lama-cleaner(now renamed to IOPaint.), partly due to the fact that during this time I have released my own first macOS applicati...

giustiroOP•7/15/24, 12:59 PM

@nerdylive If you have some recommendations about this, please do share it, would be super helpful. Thanks

Jason•7/15/24, 1:03 PM

Oh yeah i think thats fine

Deploying MIGAN model to Serverless.

Similar Threads

Similar Threads

Similar Threads