Runpod•2y ago•

60 replies

how to deploy custom image gen model on serverless?

https://blog.runpod.io/custom-models-with-serverless-and-automatic-stable-diffusion/
i read this but can i do it without the a1111? would it make it more simple to set up? i want to deploy a custom image model for image generation on serverless.

i plan on just using the sdxl API.

RunPod Blog

Create a custom AUTOMATIC1111 serverless deployment with your custo...

Have you ever wanted to create your own serverless AUTOMATIC1111 endpoint with a custom model that can scale up and down? Now you can do so without much hassle by following this guide!

Pre-requisites

A computer (local or cloud) with the following:

* docker installed
* git installed
* a relatively fast upload

Solution

For A1111, this one is better, its more up-to-date:
https://github.com/ashleykleynhans/runpod-worker-a1111

GitHub

GitHub - ashleykleynhans/runpod-worker-a1111: RunPod Serverless Wor...

RunPod Serverless Worker for the Automatic1111 Stable Diffusion API - ashleykleynhans/runpod-worker-a1111

Jump to solution

Solution

digigoblin•6/6/24, 11:57 AM

For A1111, this one is better, its more up-to-date:
https://github.com/ashleykleynhans/runpod-worker-a1111

GitHub

GitHub - ashleykleynhans/runpod-worker-a1111: RunPod Serverless Wor...

RunPod Serverless Worker for the Automatic1111 Stable Diffusion API - ashleykleynhans/runpod-worker-a1111

digigoblin•6/6/24, 11:57 AM

Otherwise for SDXL you can use this and switch out the model:
https://github.com/runpod-workers/worker-sdxl

GitHub

GitHub - runpod-workers/worker-sdxl: RunPod worker for Stable Diffu...

RunPod worker for Stable Diffusion XL. Contribute to runpod-workers/worker-sdxl development by creating an account on GitHub.

Jason•6/6/24, 12:55 PM

Yeah both works, but I think if you wanna use loras, embedding and other things on a1111, you can't do that all in sdxl worker

digigoblin•6/6/24, 1:31 PM

Yeah it just does plain inference, not even ControlNet etc

JJason Yeah both works, but I think if you wanna use loras, embedding and other things ...

BBAznOP•6/7/24, 3:52 AM

so if i want to do the loras and stuff through API i would have to go with the a1111 right, so i might as well just follow the tutorial?

BBBAzn so if i want to do the loras and stuff through API i would have to go with the a...

Jason•6/7/24, 3:52 AM

yeah, not sure if that tutorial has updated templates

Jason•6/7/24, 3:52 AM

better to use ashleyk's template

BBAznOP•6/7/24, 3:53 AM

ok ill check out the ashleyk github link

JJason better to use ashleyk's template

BBAznOP•6/8/24, 11:14 AM

im using ashleyk's image container and used my network volume that i installed the a1111 on but its like stuck on initializing?

BBAznOP•6/8/24, 11:15 AM

got to this final step in the jupyter notebook terminal when installing the a1111 on network volume

BBAznOP•6/8/24, 11:16 AM

so the network volume should be good?

BBAznOP•6/8/24, 11:16 AM

BBAznOP•6/8/24, 11:18 AM

@digigoblin

BBAznOP•6/8/24, 11:51 AM

nevermind--- i got it up, just had to switch gpu to available ones

BBAznOP•6/9/24, 1:06 AM

does cold start time billable? seems like the cold start runs up to like 2 minutes

@digigoblin @! what does this do?

BBBAzn does cold start time billable? seems like the cold start runs up to like 2 minut...

digigoblin•6/9/24, 1:06 AM

Yes it is

Ddigigoblin Yes it is

BBAznOP•6/10/24, 7:26 AM

is there access to webUI after deploying a1111 to serverless?

BBAznOP•6/10/24, 7:26 AM

or just API

digigoblin•6/10/24, 7:27 AM

Just API, Serverless is an API , its not designed for Web UI. If you want a frontend, you have to build it yourself

Ddigigoblin Just API, Serverless is an API , its not designed for Web UI. If you want a fro...

BBAznOP•6/10/24, 7:29 AM

okie, but i can just spin up a pod and open the webUI right

BBAznOP•6/10/24, 7:29 AM

i just need to check if adetailer models came installed or not

BBBAzn okie, but i can just spin up a pod and open the webUI right

digigoblin•6/10/24, 7:35 AM

Yes

BBBAzn i just need to check if adetailer models came installed or not

digigoblin•6/10/24, 7:35 AM

How did you install it in serverless? if you use this, it includes ADetailer:
https://github.com/ashleykleynhans/runpod-worker-a1111

GitHub

GitHub - ashleykleynhans/runpod-worker-a1111: RunPod Serverless Wor...

RunPod Serverless Worker for the Automatic1111 Stable Diffusion API - ashleykleynhans/runpod-worker-a1111

Ddigigoblin How did you install it in serverless? if you use this, it includes ADetailer: h...

BBAznOP•6/10/24, 7:55 AM

yes its included but i dont see any adetailer models in the folder

BBAznOP•6/10/24, 7:55 AM

leme check volume again

Ddigigoblin How did you install it in serverless? if you use this, it includes ADetailer: h...

BBAznOP•6/10/24, 8:02 AM

yeah i dont see anything in the workspace/stable-diffusion-webui/models/adetailer
directory, which i think is where models should be?

BBAznOP•6/10/24, 8:04 AM

im checking webUI

Jason•6/10/24, 8:05 AM

check the {YOURWEBUIPATH}/extensions/adetailer @BBAzn

JJason check the {YOURWEBUIPATH}/extensions/adetailer @BBAzn

BBAznOP•6/10/24, 8:11 AM

it looks like this

Jason•6/10/24, 8:11 AM

check adetailer folder

JJason check adetailer folder

BBAznOP•6/10/24, 8:13 AM

Jason•6/10/24, 8:14 AM

is it not there in your webui

BBAznOP•6/10/24, 8:15 AM

im trying to figure out how to load my UI

Jason•6/10/24, 8:15 AM

hmm im not sure but it says in the ext its should be in models/adetailer

Jason•6/10/24, 8:16 AM

download them from https://huggingface.co/Bingsu/adetailer/tree/main

Bingsu/adetailer at main

BBAznOP•6/10/24, 8:17 AM

okie

JJason hmm im not sure but it says in the ext its should be in models/adetailer

BBAznOP•6/10/24, 8:17 AM

should it be in the

workspace/stable-diffusion-webui/extensions/adetailer/adetailerworkspace/stable-diffusion-webui/extensions/adetailer/adetailer

or just

workspace/stable-diffusion-webui/extensions/adetailerworkspace/stable-diffusion-webui/extensions/adetailer

Jason•6/10/24, 8:18 AM

models path

Jason•6/10/24, 8:18 AM

stable-diffusion-webui/models

JJason models path

BBAznOP•6/10/24, 8:19 AM

workspace/stable-diffusion-webui/models/adetailerworkspace/stable-diffusion-webui/models/adetailer

Jason•6/10/24, 8:22 AM

yep should be there
/workspace/stable-diffusion-webui/models/adetailer/workspace/stable-diffusion-webui/models/adetailer

BBAznOP•6/12/24, 11:49 AM

@nerdylive if there are two API requests going to my A1111 serverless endpoint that uses two different SD models, would it be two instances of A1111? or would the A1111 instance have to swap out models? im asking just curious whether there would be a generation speed

BBAznOP•6/12/24, 11:49 AM

i mean i could test it but

BBAznOP•6/12/24, 11:49 AM

figured id ask ;D

digigoblin•6/12/24, 11:52 AM

It works fine if you use the override to set the model in the serverless request

digigoblin•6/12/24, 11:53 AM

eg:

                "override_settings":{
                    "sd_model_checkpoint":"turboDiffusionXL_v112",
                    "CLIP_stop_at_last_layers": 1
                },

                "override_settings":{
                    "sd_model_checkpoint":"turboDiffusionXL_v112",
                    "CLIP_stop_at_last_layers": 1
                },

BBAznOP•6/13/24, 7:29 AM

yes swapping seems to be no issue, but i noticed that the generation queue is lined up one by one, so when i send generation requests from VSC terminal, i would open up multiple terminals and send requests one at a time, and the serverless endpoint would generate one image, then start the other and so on

BBAznOP•6/13/24, 7:29 AM

i thought it was supposed to scale with the workers?

digigoblin•6/13/24, 7:49 AM

Depends on your endpoint configuration

BBBAzn yes swapping seems to be no issue, but i noticed that the generation queue is li...

Jason•6/13/24, 7:52 AM

You can set scale type in your endpoint to queue delay to be 1

how to deploy custom image gen model on serverless?

Similar Threads

Similar Threads

Similar Threads