Software Engineering Courses (SECourses)•7mo ago

Hi <@205854764540362752>, I cannot get either SDPA nor xformers to work with koyha_ss on my 5090. I

Hi @Furkan Gözükara SECourses, I cannot get either SDPA nor xformers to work with koyha_ss on my 5090. I have used both your install and also another method recommended by Bmaltais himself 2 weeks ago (https://github.com/bmaltais/kohya_ss/issues/3096#issuecomment-2730229439).

The installation of pytorch + xformers is successful. When I check in the .venv, the xformers are indeed installed.
However during training, the CrossAttention settings simply get ignored, both if I set it to xformers and to SDPA.
No mention of CrossAttention is shown in the training logs, and I notice on the VRAM usage that this is not being enabled.
I have all the requirements on my pc (python 3.10, cuda 12.8 etc).

Do you have any advice? Are the cross attention settings working in your koyha_ss install with your 5090, xformers in particular?
Thank you in advance.

GitHub

Is there any way to run kohya ss gui on RTX5090? · Issue #3096 · ...

Currently when I start kohya ss gui on RTX5090, I get the following error and cannot learn Lora. Maybe I should wait for an update, but is there any way to learn Lora using RTX5090? UserWarning: NV...

DDario Hi <@205854764540362752>, I cannot get either SDPA nor xformers to work with koy...

Furkan Gözükara SECourses•6/12/25, 7:18 PM

do you have python 3.10.11 installed?

Furkan Gözükara SECourses•6/12/25, 7:18 PM

i am able to use both with my installer

Furkan Gözükara SECourses•6/12/25, 7:18 PM

maybe not displaying on cmd?

Furkan Gözükara SECourses•6/12/25, 7:18 PM

did you check your speed_? which config you used?

FFurkan Gözükara SECourses do you have python 3.10.11 installed?

DarioOP•6/13/25, 8:24 AM

Thank you for your reply. Yes I have python 3.10.11, I am using the latest config "48GB_GPU_28200MB_6.3_second_it_Tier_1".
Speed is around 3.85s/it.
Perhaps it is indeed working but I find it strange there is no menion of this in the cmd windows. I remember the use of xformers was shown here on older installs (when using a 4090).
Do you see this in your koyha cmd windows when training?

FFurkan Gözükara SECourses do you have python 3.10.11 installed?

DarioOP•6/13/25, 8:28 AM

Further more, I am getting the exact same 3.85s/it speed and 27.7 GB VRAM used when I set CrossAttention to "none" in tge Koyha GUI, as when I set it to "xformers".
So that shows that there is no cross attention happening.

JJamiewhelton is like specky-ness everywhere. I think it only happens after refining.

Timson•6/13/25, 10:00 AM

my first guesst would be - some compression artefacts trained in with dataset. Another potential issue - you are trying to generate at much larger resolution than you training set

Timson•6/13/25, 10:01 AM

I was able to partially work aroung models trained on subpar datasets by doing img2img from donor image of high quality and fairly large creativity

Timson•6/13/25, 10:03 AM

It would be cool if someone found prompting techniques to improve quality (the obvious ones like "hi res image", "high quality image" have almost no effect on the generation)

DDario Further more, I am getting the exact same 3.85s/it speed and 27.7 GB VRAM used w...

Furkan Gözükara SECourses•6/13/25, 11:14 AM

3.85 is accurate

Furkan Gözükara SECourses•6/13/25, 11:14 AM

it is working

Furkan Gözükara SECourses•6/13/25, 11:14 AM

also SPDA and xFormers are same speed recently

FFurkan Gözükara SECourses 3.85 is accurate

DarioOP•6/13/25, 11:29 AM

Okay but if I set cross attention to "none" I get the sane speed and VRAM. That and the missing mention of this in the cmd window makes me very suspicious if this is actually working.

DDario Okay but if I set cross attention to "none" I get the sane speed and VRAM. That ...

Furkan Gözükara SECourses•6/13/25, 11:50 AM

ah i think you should report it

Furkan Gözükara SECourses•6/13/25, 11:50 AM

because that shouldnt happen

FFurkan Gözükara SECourses ah i think you should report it

DarioOP•6/13/25, 12:06 PM

Okay I will do that, thanks.

DDario Okay I will do that, thanks.

DarioOP•6/13/25, 1:33 PM

Just to cross off all posibilities; which nvidia driver do you have installed?

Jamiewhelton•6/13/25, 2:46 PM

can I train at higher resolution than 1024x1024 with this setup... dreambooth flux

JJamiewhelton can I train at higher resolution than 1024x1024 with this setup... dreambooth fl...

Furkan Gözükara SECourses•6/13/25, 10:00 PM

yes but i didnt see much benefit

Original message was deleted

Furkan Gözükara SECourses•6/13/25, 10:00 PM

removing it from dataset

Furkan Gözükara SECourses•6/13/25, 10:01 PM

or in prompt after training try to prompt carefully that would generate different side eye

Punceka•6/14/25, 9:20 AM

Is there any way to resume LoRa training after the overnight automatic Windows update that restarted my PC?

LukeJoseph•6/14/25, 6:51 PM

Hi all - followed the Flux training Lora guide/tutorial. Does my speeds seem correct for a 3090? It’s roughly 9 it/s using the level 2 workflow.

I keep seeing references to a “massive speed increase” with torch 2.5, but the 1 click installer has I believe installs 2.7? So I imagine that’s already implemented.

FluxGym is substantially faster, around 2 it/s, if memory serves me. But as I am just learning about all this, is a bit difficult for me to compare apples to apples/settings to settings. FluxGym while easy to use, doesn’t have the most friendly of interfaces when digging into additional settings.

! Pixdls•6/14/25, 9:41 PM

how could i fix this?

PPunceka Is there any way to resume LoRa training after the overnight automatic Windows u...

Furkan Gözükara SECourses•6/14/25, 10:22 PM

yes

Furkan Gözükara SECourses•6/14/25, 10:22 PM

tips-info

LLukeJoseph Hi all - followed the Flux training Lora guide/tutorial. Does my speeds seem cor...

Furkan Gözükara SECourses•6/14/25, 10:22 PM

yes already implemented

Furkan Gözükara SECourses•6/14/25, 10:22 PM

FluxGym is not faster using different config

Furkan Gözükara SECourses•6/14/25, 10:23 PM

and a weaker one

!! Pixdls how could i fix this?

Furkan Gözükara SECourses•6/14/25, 10:23 PM

model download looks like failed / missing

Furkan Gözükara SECourses•6/14/25, 10:23 PM

hugging face is down atm

FFurkan Gözükara SECourses hugging face is down atm

! Pixdls•6/14/25, 10:44 PM

ah okay

! Pixdls•6/14/25, 10:45 PM

thanks for the reply, i've managed to fix it but now i run into this. Can this been the file is corrupted?

!! Pixdls thanks for the reply, i've managed to fix it but now i run into this. Can this b...

Furkan Gözükara SECourses•6/14/25, 10:50 PM

yep can be corrupted

FFurkan Gözükara SECourses yes already implemented

LukeJoseph•6/14/25, 11:24 PM

That’s what I figured. Appreciate the response!

LLukeJoseph That’s what I figured. Appreciate the response!

Furkan Gözükara SECourses•6/14/25, 11:28 PM

you are welcome

Furkan Gözükara SECourses•6/14/25, 11:28 PM

serializing messages usually related to model corruption

Jamiewhelton•6/15/25, 6:28 AM

Scammer

JJamiewhelton Scammer

Furkan Gözükara SECourses•6/15/25, 10:25 AM

yep banned

PabloMoises•6/17/25, 1:58 PM

Hi! I have a question, maybe someone can help me.

Can we retrain a fine-tuned model to create another fine-tuned model based on it instead of starting from Flux1dev?

Thanks!

Smagash•6/17/25, 3:42 PM

What’s the best program or extension to use right now to make a sd model of myself

Smagash•6/17/25, 3:46 PM

I will take any recommendations

PPabloMoises Hi! I have a question, maybe someone can help me. Can we retrain a fine-tuned m...

Furkan Gözükara SECourses•6/17/25, 10:08 PM

yes but it gets worse as you train also if same concept it will bleed

SSmagash What’s the best program or extension to use right now to make a sd model of myse...

Furkan Gözükara SECourses•6/17/25, 10:08 PM

onetrainer

Furkan Gözükara SECourses•6/17/25, 10:09 PM

https://youtu.be/0t5l6CP9eBg

YouTubeSECourses

Full Stable Diffusion SD & XL Fine Tuning Tutorial With OneTrainer ...

In this tutorial, I am going to show you how to install OneTrainer from scratch on your computer and do a Stable Diffusion SDXL (Full Fine-Tuning 10.3 GB VRAM) and SD 1.5 (Full Fine-Tuning 7GB VRAM) based models training on your computer and also do the same training on a very cheap cloud machine from MassedCompute if you don't have such compute...

�

🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭•6/18/25, 12:13 AM

I trained a flux lora on cicitai, and for testing i went with a real person. On generations, skin gets plastic and shiny...not like my dataset. Using heun beta 30 steps. Curious about what might cause this

�🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭I trained a flux lora on cicitai, and for testing i went with a real person. On ...

Furkan Gözükara SECourses•6/18/25, 10:34 AM

civitai training weak

Furkan Gözükara SECourses•6/18/25, 10:34 AM

see my latest tutorial i shown myself trained result

Furkan Gözükara SECourses•6/18/25, 10:35 AM

https://youtu.be/Xbn93GRQKsQ

YouTubeSECourses

WAN 2.1 FusionX is the New Best of Local Video Generation with Only...

FusionX: The BEST AI Video Model? + FLUX Hyper-Realistic Upscaling (One-Click Setup!). Struggling to create high-quality AI videos and hyper-realistic images? This tutorial is your ultimate solution! I'm introducing the incredible new Wan 2.1 FusionX model and a game-changing 2x latent upscaler for the FLUX model, all made incredibly simple with...

Furkan Gözükara SECourses•6/18/25, 10:35 AM

13:48 Generating a High-Quality Image with The Official FLUX Preset
14:50 Using Automatic Face Segmentation & Inpainting with FLUX
16:05 The Ultimate Upgrade: Applying The FLUX 2x Latent Upscaler Preset
16:32 Final Result: Comparing Standard vs. 2x Upscaled Image Quality
16:50 Outro & Sneak Peek of The New Ultimate Video Processing App

Hi <@205854764540362752>, I cannot get either SDPA nor xformers to work with koyha_ss on my 5090. I

Similar Threads