Best tips for lowering SDXL text2image API startup latency?
I'm currently using the https://github.com/ashleykleynhans/runpod-worker-a1111 along with a network volume. I only use a single model with the sd text2image endpoint and I don't need the UI. Right now, I'm experiencing an 80+ second delay for cold startups on the first request. Do you have any suggestions on how to optimize this (without 1 constant active worker)? Thanks in advance!
9 Replies
Or is it better to run comfyui api, is that one faster to boot?
Unknown User•13mo ago
Message Not Public
Sign In & Join Server To View
Ok, maybe not then! 😅
Unknown User•13mo ago
Message Not Public
Sign In & Join Server To View
Do you know how long it takes for it to become cold?
Unknown User•13mo ago
Message Not Public
Sign In & Join Server To View
Alright, thanks!
I'm also experimenting with both as well. Some says the long latency may also caused by using network volume. I haven't really tried package models in the docker image myself as I kind of need a lot of models and loras, not sure if it's worth packaging in the docker image
Let me know if you find out something useful 🙂 For now I'm keeping the endpoint warm by sending a request every 2 minutes