Understanding serverless & prising. Usecase: a1111 --api w. control on serverless
Right now im running some version of juggernautxl w controlnet, i spend about 20 seconds on generating 1 images, sometimes less on a5000. However, sometimes my reponse time on my endpoints are very slow. Im trying to figure out if this is purely because of cold start time, or because my requests is in the que before landing on a GPU. I guess my question is:
-How to see when GPU is in que vs cold start, and is que time billed? How to control que time?
-Does anyone have experience in reducing cold start time for a1111 serverless requests, and or maybe have setup some difffuser setup without a1111 for serverless that function really good?
-If you can help me AWESOME. im shit, but my project is really cool.
-How to see when GPU is in que vs cold start, and is que time billed? How to control que time?
-Does anyone have experience in reducing cold start time for a1111 serverless requests, and or maybe have setup some difffuser setup without a1111 for serverless that function really good?
-If you can help me AWESOME. im shit, but my project is really cool.
