Runpod•2y ago

how to deploy custom image gen model on serverless?

https://blog.runpod.io/custom-models-with-serverless-and-automatic-stable-diffusion/ i read this but can i do it without the a1111? would it make it more simple to set up? i want to deploy a custom image model for image generation on serverless. i plan on just using the sdxl API.

RunPod Blog

Create a custom AUTOMATIC1111 serverless deployment with your custo...

Have you ever wanted to create your own serverless AUTOMATIC1111 endpoint with a custom model that can scale up and down? Now you can do so without much hassle by following this guide! Pre-requisites A computer (local or cloud) with the following: * docker installed * git installed * a relatively fast upload

Solution:

For A1111, this one is better, its more up-to-date: https://github.com/ashleykleynhans/runpod-worker-a1111...

GitHub

GitHub - ashleykleynhans/runpod-worker-a1111: RunPod Serverless Wor...

RunPod Serverless Worker for the Automatic1111 Stable Diffusion API - ashleykleynhans/runpod-worker-a1111

Jump to solution

39 Replies

Solution

digigoblin•2y ago

For A1111, this one is better, its more up-to-date: https://github.com/ashleykleynhans/runpod-worker-a1111

GitHub

GitHub - ashleykleynhans/runpod-worker-a1111: RunPod Serverless Wor...

RunPod Serverless Worker for the Automatic1111 Stable Diffusion API - ashleykleynhans/runpod-worker-a1111

digigoblin•2y ago

Otherwise for SDXL you can use this and switch out the model: https://github.com/runpod-workers/worker-sdxl

GitHub

GitHub - runpod-workers/worker-sdxl: RunPod worker for Stable Diffu...

RunPod worker for Stable Diffusion XL. Contribute to runpod-workers/worker-sdxl development by creating an account on GitHub.

Unknown User•2y ago

Message Not Public

digigoblin•2y ago

Yeah it just does plain inference, not even ControlNet etc

BBAznOP•2y ago

so if i want to do the loras and stuff through API i would have to go with the a1111 right, so i might as well just follow the tutorial?

Unknown User•2y ago

Message Not Public

BBAznOP•2y ago

ok ill check out the ashleyk github link im using ashleyk's image container and used my network volume that i installed the a1111 on but its like stuck on initializing?

BBAznOP•2y ago

got to this final step in the jupyter notebook terminal when installing the a1111 on network volume

BBAznOP•2y ago

so the network volume should be good?

BBAznOP•2y ago

@digigoblin nevermind--- i got it up, just had to switch gpu to available ones does cold start time billable? seems like the cold start runs up to like 2 minutes 🥶 @digigoblin @! what does this do?

digigoblin•2y ago

Yes it is

BBAznOP•2y ago

is there access to webUI after deploying a1111 to serverless? or just API

digigoblin•2y ago

Just API, Serverless is an API , its not designed for Web UI. If you want a frontend, you have to build it yourself

BBAznOP•2y ago

okie, but i can just spin up a pod and open the webUI right i just need to check if adetailer models came installed or not

digigoblin•2y ago

Yes

digigoblin•2y ago

How did you install it in serverless? if you use this, it includes ADetailer: https://github.com/ashleykleynhans/runpod-worker-a1111

GitHub

GitHub - ashleykleynhans/runpod-worker-a1111: RunPod Serverless Wor...

RunPod Serverless Worker for the Automatic1111 Stable Diffusion API - ashleykleynhans/runpod-worker-a1111

BBAznOP•2y ago

yes its included but i dont see any adetailer models in the folder leme check volume again yeah i dont see anything in the workspace/stable-diffusion-webui/models/adetailer directory, which i think is where models should be? im checking webUI

Unknown User•2y ago

Message Not Public

BBAznOP•2y ago

it looks like this

Unknown User•2y ago

Message Not Public

BBAznOP•2y ago

Unknown User•2y ago

Message Not Public

BBAznOP•2y ago

im trying to figure out how to load my UI

Unknown User•2y ago

Message Not Public

BBAznOP•2y ago

okie should it be in the workspace/stable-diffusion-webui/extensions/adetailer/adetailer or just workspace/stable-diffusion-webui/extensions/adetailer

Unknown User•2y ago

Message Not Public

BBAznOP•2y ago

workspace/stable-diffusion-webui/models/adetailer

Unknown User•2y ago

Message Not Public

BBAznOP•2y ago

@nerdylive if there are two API requests going to my A1111 serverless endpoint that uses two different SD models, would it be two instances of A1111? or would the A1111 instance have to swap out models? im asking just curious whether there would be a generation speed i mean i could test it but figured id ask ;D

digigoblin•2y ago

It works fine if you use the override to set the model in the serverless request eg:

                "override_settings":{
                    "sd_model_checkpoint":"turboDiffusionXL_v112",
                    "CLIP_stop_at_last_layers": 1
                },

                "override_settings":{
                    "sd_model_checkpoint":"turboDiffusionXL_v112",
                    "CLIP_stop_at_last_layers": 1
                },

BBAznOP•2y ago

yes swapping seems to be no issue, but i noticed that the generation queue is lined up one by one, so when i send generation requests from VSC terminal, i would open up multiple terminals and send requests one at a time, and the serverless endpoint would generate one image, then start the other and so on i thought it was supposed to scale with the workers?

digigoblin•2y ago

Depends on your endpoint configuration

Unknown User•2y ago

Message Not Public

BBAznOP•2y ago

oh ok, should i change the request count?

Unknown User•2y ago

Message Not Public

BBAznOP•2y ago

oh ok its working

BBAznOP•17mo ago

@digigoblin hey guys so i deleted my previous volume and did a fresh reinstall but now the adetailer wont trigger through API ;-; i have the adetailer models in /workspace/stable-diffusion-webui/models/adetailer and i was able to get it working in the webui on a pod, but the API just doesnt seem to pick it up? my code is the same too ;-;

BBAznOP•17mo ago

i did a GET extensions call and i saw the adetailer extension is there so the adetailer is installed i mean i was able to get the adetailer working in the webui so its definitely installed lol nvm i figured it out, it had something to do with an adetailer config "ad_tab_enable": true, after removing that it was good

Gaming

Programming

how to deploy custom image gen model on serverless?

Did you find this page helpful?