oh nvm I just relogged in hah
oh nvm I just relogged in hah
Fbase model could be reason too
Fby the way i dont have good sdxl lora config right now
Fi recommend dreambooth
Fcould be out of RAM
Fok i am glad it wasnt out of RAM 
Myes me too haha, thank you regardless
JCan someone help me diagnose this issue. Images from my recent fine tuned model are coming out with strange patterns particularly noticable on skin but also just overall in the image. Is it due to over training or could it be from particular grainy images in dataset?
Jis like specky-ness everywhere. I think it only happens after refining.
JI think it might be due to the lack of distant photos I trained with... as I used mostly high detailed mostly selfies.
MI am also curious why this is happening
�if your dataset has grainy images than it's because of that. If not then your learning rate is high. If your images are grainy try reducing the lora rank and re train
JI will try that thank you
Fcould be dataset
Mmay I ask why it says that the total amount of epochs will be 195 by the end of the training, I am sure that I selected 200
Mthis happened yesterday too when I trained another fine-tuned model, I selected 200 epochs but it did 181 I think if I remember correctly
Fbatch size number of images
Fdont worry
MOkay then
DHi @Furkan Gözükara SECourses, I cannot get either SDPA nor xformers to work with koyha_ss on my 5090. I have used both your install and also another method recommended by Bmaltais himself 2 weeks ago (https://github.com/bmaltais/kohya_ss/issues/3096#issuecomment-2730229439).
The installation of pytorch + xformers is successful. When I check in the .venv, the xformers are indeed installed.
However during training, the CrossAttention settings simply get ignored, both if I set it to xformers and to SDPA.
No mention of CrossAttention is shown in the training logs, and I notice on the VRAM usage that this is not being enabled.
I have all the requirements on my pc (python 3.10, cuda 12.8 etc).
Do you have any advice? Are the cross attention settings working in your koyha_ss install with your 5090, xformers in particular?
Thank you in advance.
The installation of pytorch + xformers is successful. When I check in the .venv, the xformers are indeed installed.
However during training, the CrossAttention settings simply get ignored, both if I set it to xformers and to SDPA.
No mention of CrossAttention is shown in the training logs, and I notice on the VRAM usage that this is not being enabled.
I have all the requirements on my pc (python 3.10, cuda 12.8 etc).
Do you have any advice? Are the cross attention settings working in your koyha_ss install with your 5090, xformers in particular?
Thank you in advance.
GitHub
Currently when I start kohya ss gui on RTX5090, I get the following error and cannot learn Lora. Maybe I should wait for an update, but is there any way to learn Lora using RTX5090? UserWarning: NV...
Fdo you have python 3.10.11 installed?
Fi am able to use both with my installer
Fmaybe not displaying on cmd?
Fdid you check your speed_? which config you used?
DThank you for your reply. Yes I have python 3.10.11, I am using the latest config "48GB_GPU_28200MB_6.3_second_it_Tier_1".
Speed is around 3.85s/it.
Perhaps it is indeed working but I find it strange there is no menion of this in the cmd windows. I remember the use of xformers was shown here on older installs (when using a 4090).
Do you see this in your koyha cmd windows when training?
Speed is around 3.85s/it.
Perhaps it is indeed working but I find it strange there is no menion of this in the cmd windows. I remember the use of xformers was shown here on older installs (when using a 4090).
Do you see this in your koyha cmd windows when training?
DFurther more, I am getting the exact same 3.85s/it speed and 27.7 GB VRAM used when I set CrossAttention to "none" in tge Koyha GUI, as when I set it to "xformers".
So that shows that there is no cross attention happening.
So that shows that there is no cross attention happening.
Tmy first guesst would be - some compression artefacts trained in with dataset. Another potential issue - you are trying to generate at much larger resolution than you training set
TI was able to partially work aroung models trained on subpar datasets by doing img2img from donor image of high quality and fairly large creativity
TIt would be cool if someone found prompting techniques to improve quality (the obvious ones like "hi res image", "high quality image" have almost no effect on the generation)
F3.85 is accurate
Fit is working
Falso SPDA and xFormers are same speed recently
DOkay but if I set cross attention to "none" I get the sane speed and VRAM. That and the missing mention of this in the cmd window makes me very suspicious if this is actually working.
Fah i think you should report it
Fbecause that shouldnt happen
DOkay I will do that, thanks.
DJust to cross off all posibilities; which nvidia driver do you have installed?
Jcan I train at higher resolution than 1024x1024 with this setup... dreambooth flux
Fyes but i didnt see much benefit
Fremoving it from dataset
For in prompt after training try to prompt carefully that would generate different side eye
PIs there any way to resume LoRa training after the overnight automatic Windows update that restarted my PC?
LHi all - followed the Flux training Lora guide/tutorial. Does my speeds seem correct for a 3090? It’s roughly 9 it/s using the level 2 workflow.
I keep seeing references to a “massive speed increase” with torch 2.5, but the 1 click installer has I believe installs 2.7? So I imagine that’s already implemented.
FluxGym is substantially faster, around 2 it/s, if memory serves me. But as I am just learning about all this, is a bit difficult for me to compare apples to apples/settings to settings. FluxGym while easy to use, doesn’t have the most friendly of interfaces when digging into additional settings.
I keep seeing references to a “massive speed increase” with torch 2.5, but the 1 click installer has I believe installs 2.7? So I imagine that’s already implemented.
FluxGym is substantially faster, around 2 it/s, if memory serves me. But as I am just learning about all this, is a bit difficult for me to compare apples to apples/settings to settings. FluxGym while easy to use, doesn’t have the most friendly of interfaces when digging into additional settings.
!how could i fix this?
F
Fyes already implemented
FFluxGym is not faster using different config
Fand a weaker one
Fmodel download looks like failed / missing
Fhugging face is down atm