R
RunPod2mo ago
BBAzn

runpod-worker-a1111 and loras

I dont think my loras are working with this worker? But it seems to be able to get loras with the /sdapi/va/loras https://github.com/ashleykleynhans/runpod-worker-a1111/blob/main/docs/api/a1111/get-loras.md so am i able to use loras with this worker or no?
GitHub
runpod-worker-a1111/docs/api/a1111/get-loras.md at main · ashleykle...
RunPod Serverless Worker for the Automatic1111 Stable Diffusion API - ashleykleynhans/runpod-worker-a1111
76 Replies
BBAzn
BBAzn2mo ago
actually you know what i did the GET /sdapi/va/loras and the output came back empty? i think i might have to put the loras in the wrong place? they are currently in the /workspace/stable-diffusion-webui/models/Lora#
nerdylive
nerdylive2mo ago
V1? Is that the right endpoint payload Yeah it should work
digigoblin
digigoblin2mo ago
You must have put the LoRA in the wrong folder if its empty Maybe you also need to call the refresh checkpoints endpoint
BBAzn
BBAzn2mo ago
this is the folder
No description
digigoblin
digigoblin2mo ago
why is lora lowercase Linux is case sensitive
BBAzn
BBAzn2mo ago
i just changed it lowercase it was uppercase before should it be uppercase?
digigoblin
digigoblin2mo ago
Leave it the way it was before, it won't work if you change it because Linux is case sensitive
nerdylive
nerdylive2mo ago
I think it's that way for a1 Oh nvm I don't remember it so well
BBAzn
BBAzn2mo ago
it was always uppercase but since it dont work i changed it but i changed it back to uppercase anyways let me do a refresh
digigoblin
digigoblin2mo ago
stable-diffusion-webui/models/Lora
stable-diffusion-webui/models/Lora
thats how it should be
BBAzn
BBAzn2mo ago
yeah it didnt detect it btw im on serverless endpoint
digigoblin
digigoblin2mo ago
Sorry not refresh checkpoints, thats for models
digigoblin
digigoblin2mo ago
digigoblin
digigoblin2mo ago
refresh-loras
BBAzn
BBAzn2mo ago
yes /sdapi/v1/refresh-loras { "delayTime": 463, "error": "A1111 status code: 405", "executionTime": 84, "id": "c95c0ac3-86e1-4c22-b6da-07dbff387401-u1", "output": { "output": { "detail": "Method Not Allowed" } }, "status": "FAILED" }
digigoblin
digigoblin2mo ago
What payload did you use?
BBAzn
BBAzn2mo ago
nvm lol i did get lmao hold on waiting in queue btw i get coldstarts alot even when im like constantly generating the GPUs seems to get taken away from my serverless randomly
digigoblin
digigoblin2mo ago
Did you enable flash boot on your endpoint?
BBAzn
BBAzn2mo ago
No description
BBAzn
BBAzn2mo ago
yes { "delayTime": 63844, "executionTime": 60, "id": "00330fa9-ac79-4a2d-aa10-3df3a5eede64-u1", "output": [], "status": "COMPLETED" } its still not detecting my loras...do i have to name them with an extension? like add-detail-xl.tensorflow or something
digigoblin
digigoblin2mo ago
not .tensorflow, .safetensors but yes, they should typically have an extension like .safetensors, .ckpt etc
BBAzn
BBAzn2mo ago
also is there a way to refresh the serverless endpoint with the new data in the network volume? i find myself constantly deleting the serverless and remaking it when i make changes to the network volume
digigoblin
digigoblin2mo ago
You shouldn't have to delete it and recreate it, it should update most things automatically, you may just need to call the refresh endpoints
BBAzn
BBAzn2mo ago
ohh right silly me ok that worked lol the loras are there i just didnt have the extensions for them lmao
BBAzn
BBAzn2mo ago
thx ❤️
digigoblin
digigoblin2mo ago
Great stuff
nerdylive
nerdylive2mo ago
Huh u can use it from the prompt
BBAzn
BBAzn2mo ago
i didnt name the lora files with the .safetensors extension in the folder so the GET lora endpoint wasnt picking them up they definitely working now
nerdylive
nerdylive2mo ago
Ooh Okok files ext
digigoblin
digigoblin2mo ago
Yeah all model types need a file extension or they won't be found, whether its SD, ControlNet, LoRA etc
BBAzn
BBAzn2mo ago
yeah i knew that too cuz i had the same problem with models and then i renamed them and they was found
digigoblin
digigoblin2mo ago
Probably to do with the way models are loaded, for example ckpt files can have malware embedded into them (which is why people switched to using .safetensors instead) so need to know the filename to determine how to load it, because if the extension is .ckpt, safe unpickle can be used to avoid malware from being loaded into the system.
digigoblin
digigoblin2mo ago
I typically try to avoid .ckpt models and use .safetensors only
BBAzn
BBAzn2mo ago
yeah i havnt came across any ckpt yet
digigoblin
digigoblin2mo ago
Most people switched away from it luckily
BBAzn
BBAzn2mo ago
do you guys know if there is an A1111 API schema somewhere? like a complete one? for example i found out that i need to set Emphasis: No norm but dunno what the API setting is and its not listed in the ashley wiki woop nvm i found a pdf
nerdylive
nerdylive2mo ago
That one might not be up to date
BBAzn
BBAzn2mo ago
it doesnt list it anyways
nerdylive
nerdylive2mo ago
I forgot wheres the a1111 docs is
BBAzn
BBAzn2mo ago
yeah its from 2023
nerdylive
nerdylive2mo ago
When you run one in like pods, it has one endpoint for the docs as html
BBAzn
BBAzn2mo ago
yeah
nerdylive
nerdylive2mo ago
Like from the webui
BBAzn
BBAzn2mo ago
leme try one just set it up as the pod
nerdylive
nerdylive2mo ago
Right
BBAzn
BBAzn2mo ago
RunPod Fast Stable Diffusion runpod/stable-diffusion:fast-stable-diffusion-2.4.0 template
digigoblin
digigoblin2mo ago
Not sure whether that one has --api enabled If --api is not enabled, you can't access the API docs
BBAzn
BBAzn2mo ago
i found it its "emphasis": "Original", POST /sdapi/v1/options Set Config inside there
nerdylive
nerdylive2mo ago
What's that for hahah
BBAzn
BBAzn2mo ago
so i spun up a pod and then just accessed the /docs
nerdylive
nerdylive2mo ago
Never heard of it Right click, save as html so you don't have to memorize it lol
BBAzn
BBAzn2mo ago
i dont even know all i know is it stops pony xl from crashing not crashing but
BBAzn
BBAzn2mo ago
No description
BBAzn
BBAzn2mo ago
there are so many API config options lol
nerdylive
nerdylive2mo ago
Oh is that a1111 Oh the Emphasize normalization yeah yeah
BBAzn
BBAzn2mo ago
yeah its A1111 the option in the webui nice now i have all the APis lol
nerdylive
nerdylive2mo ago
Hahah yeah Gonna be reading that for a day? Lol Alot of options
BBAzn
BBAzn2mo ago
yeah read all and then i wont have any issues when ppl talk about some setting im like oh im on API i dunno if i can do it lol do you guys know why everytime i make a request its a cold start? and it takes like 4-5 minutes per generation
BBAzn
BBAzn2mo ago
No description
BBAzn
BBAzn2mo ago
2024-06-14 16:17:57.805 [sjvl6zqmehakh0] [info] Service not ready yet. Retrying... its like everytime im done gen the gpu gets taken away
nerdylive
nerdylive2mo ago
Both in the same like minute?
BBAzn
BBAzn2mo ago
yeah once its done i usually send another request immediately right after
BBAzn
BBAzn2mo ago
No description
BBAzn
BBAzn2mo ago
it is low availability right now so i dunno i usually pick 2 but today im picking all of them except the 80s
nerdylive
nerdylive2mo ago
Oh yeah maybe try again Did you have refresh worker? Cold starts are random
BBAzn
BBAzn2mo ago
wat is refresh worker
nerdylive
nerdylive2mo ago
Oh nvm then its just random, flashboot isnt activated all the time, only if you have continous requests i think
BBAzn
BBAzn2mo ago
yeah its random sometimes its fast even after some time in between
nerdylive
nerdylive2mo ago
im not sure what happened so maybe the flashboot wasn't activated or maybe your worker got throttled because of the low availibility but flashboot is kinda random for low amount request ig
BBAzn
BBAzn2mo ago
pretty sure its low availability, im considering copying pasting the network volume to a different timezone but they dont seem to have much availability either from the preview, are you guys gonna add more gpu lol
nerdylive
nerdylive2mo ago
ofc runpod will eventually unless something happened