Topics

RunPod•4mo ago

Stable Diffusion GPU Pod and API

Is there a way to connect a GPU Pod running the stable diffusion template to an API layer that is externally exposed.? I have a serverless instance running @ashleyk 's docker which is working great and much appreciated, albeit 10x slower than the GPU Pods. I am attempting to leverage the processing power and number of GPUs on the pod side -- but need an API endpoint that I can expose to my external app... @flash-singh ? @justin I appreciate your answers, but I am directing this to the RunPod support or @ashleyk if willing to chip in.

4 Replies

ashleyk•4mo ago

You can set active workers or increase your idle timeout to speed up serverless. You can't use A1111 API with GPU Cloud in production because it doesn't scale, so either use serverless A1111 or if you want to optimize speed, look at using diffusers instead of A1111.

KJK•4mo ago

Thanks Ashley. Even with active workers, I am getting 10 times the response time than what I get on GPU Pods. I'll look at the diffusers.

flash-singh•4mo ago

you need to run same codebase to get similar speeds, theres many packages that offer speeds and its a optimization issue most of the time

ashleyk•4mo ago

Yeah, should maybe consider using Forge instead of A1111.

We're a community of enthusiasts, engineers, and enterprises, all sharing insights on AI, Machine Learning and GPUs!

11KMembers

View on Discord

Want results from more Discord servers?

Add your server

More Posts

Is there a way to access worker ID & job ID from a handler? Would be good for logging + debuggingIf Runpod could drop worker ID and job ID into environment variables made available to handlers that Horrible network speeds make the pod unusable.This happens regularly. there where several posts about this a few months back. In one it was sugge How can I deploy Mixtral using Ollama as service?Hi everyone! I want deploy mixtral 7x8b model using ollama on runpod, but I can't install it as serv 520: Web server is returning an unknown errorI'm getting a persistent error trying to connect to our API: <title>PODID-XXXX.proxy.runpod.net | 52 Driver mismatchI've created a pod with NVIDIA-SMI 535.154.05 Driver Version: 535.154.05 CUDA Version: 12.2 NVM Servers' availability: "Any" region vs Specific regionsHi, I'm trying to rent a server with H100 in secure cloud. When I choose "Any" option in the region,File copying does not occur in Custom TemplateI have been successfully using runpod with a custom template until recently when it stopped working Serverless errors in the logsI am having a lot of these errors in our logs lately. In our app we have nothing that indicates a pr Having trouble with Serverless SD XL imageThere is no image that gets generated and I only get a success notification and no error. Unable to What does "Low Availability" mean?Some instances have the tag of "Low Availability". I'm wondering what does that exactly entail?