Software Engineering Courses (SECourses)•5mo ago

Since I've been deep in training with the command line, I had a few thoughts on what might be needed

Since I've been deep in training with the command line, I had a few thoughts on what might be needed to get it fully working for models like Qwen .

The most important thing would be a dropdown at the top to choose the trainer script, like for qwen_image_train_network.py or flux_kontext_train_network.py, since they use diferent scripts.
In the Advanced tab, I definately needed options for the attention mechanism (--sdpa / --xformers) and the fp8_base / fp8_scaled toggles.

Those were super important to get my training running on a 24GB card without errors.

Also, the Network Module dropdown needs to have the specific options for each model, like networks.lora_qwen_image. My training failed until I found the right one for Qwen.

One small thing, maybe the precision setting (bf16/fp16) could be a dropdown insted of checkboxes?

Might be less confusing.
Just some ideas from my recent training hell lol.

The layout is great though, really clean. Keep up the awesome work!

Ccyberbol Since I've been deep in training with the command line, I had a few thoughts on ...

Furkan Gözükara SECourses•8/14/25, 3:50 PM

i am simplifying

Furkan Gözükara SECourses•8/14/25, 3:50 PM

qwen tab only qwen

FFurkan Gözükara SECourses qwen tab only qwen 😄

cyberbolOP•8/14/25, 3:51 PM

Ok. But always open gate to add more functionality later xd

Furkan Gözükara SECourses•8/14/25, 3:51 PM

Furkan Gözükara SECourses•8/14/25, 3:51 PM

Furkan Gözükara SECourses•8/14/25, 3:51 PM

FFurkan Gözükara SECourses Click to see attachment

cyberbolOP•8/14/25, 3:56 PM

Here are the most critical things:
FP8 Options: We definately need two checkboxes in the "Advanced" tab: one for --fp8_base and one for --fp8_scaled. These were the biggest VRAM savers and the training wouldn't even start without them.
Blocks to Swap: Also in "Advanced", we need an input box for --blocks_to_swap. For a 1024x1024 training on a 24GB card, I had to use a value of 20 to prevent crashes.
Attention Mode: A dropdown to select the attention mechanism is also essential.

My training only worked after I set it to --sdpa.
Network Module: This one is critical! In the "LoRA Parameters" tab, the dropdown for Network Module absolutely needs to have networks.lora_qwen_image as an option. The generic networks.lora module made my training crash with an error.

One other small idea, maybe the Precision setting in the "Source Model" tab could be a dropdown menu (bf16, fp16) insted of separate checkboxes?

Might prevent users from accidentally clicking more than one.

For bonus points, adding fields for advanced stuff like LoRA+ (the network_args field you have is perfect for this!) and PyTorch Dynamo (--dynamo_backend) would make this the ultimate tool.

Hope this feedback from my command-line struggles is helpful! The GUI is already looking amazing. Can't wait to try it when it's ready. Keep up the awesome work!

Furkan Gözükara SECourses•8/14/25, 3:57 PM

FFurkan Gözükara SECourses Click to see attachment

cyberbolOP•8/14/25, 3:58 PM

You think about everything

cyberbolOP•8/14/25, 3:58 PM

Boss

cyberbolOP•8/14/25, 3:58 PM

I learned a lot from you btw.

FFurkan Gözükara SECourses Click to see attachment

JonkoXL•8/14/25, 4:23 PM

network dim , network alpha, vae_cache cpu?

JonkoXL•8/14/25, 4:24 PM

next thing you know , you rewrote koyha_ss gui

AiInfluence•8/14/25, 4:24 PM

this is just amazing

AiInfluence•8/14/25, 4:24 PM

the doctor

AiInfluence•8/14/25, 4:24 PM

master of neural thinking

AiInfluence•8/14/25, 4:24 PM

DIM NETWORKS

AiInfluence•8/14/25, 4:24 PM

jusst amazing

JonkoXL•8/14/25, 4:24 PM

neural thinking > neurotic thinking

JJonkoXL network dim , network alpha, vae_cache cpu?

Furkan Gözükara SECourses•8/14/25, 5:03 PM

Furkan Gözükara SECourses•8/14/25, 5:03 PM

Furkan Gözükara SECourses•8/14/25, 5:03 PM

Furkan Gözükara SECourses•8/14/25, 5:04 PM

i will also add a feature to auto remember and reload last used config

cyberbolOP•8/14/25, 5:15 PM

Caching will be automatically before trening process or we will need do do separate?

FFurkan Gözükara SECourses i will also add a feature to auto remember and reload last used config

JonkoXL•8/14/25, 5:23 PM

Awesome

JonkoXL•8/14/25, 5:23 PM

I am impressed

JonkoXL•8/14/25, 5:23 PM

Will you make it generic or easily adaptable so you can reuse it for Wan 2.2 training?

Ccyberbol Caching will be automatically before trening process or we will need do do separ...

Furkan Gözükara SECourses•8/14/25, 5:28 PM

good question i need to ensure it happens auto

Furkan Gözükara SECourses•8/14/25, 5:28 PM

when selected

JJonkoXL Will you make it generic or easily adaptable so you can reuse it for Wan 2.2 tra...

Furkan Gözükara SECourses•8/14/25, 5:28 PM

yes i plan to add wan 2.2 as well

Furkan Gözükara SECourses•8/14/25, 5:28 PM

after qwen done

FFurkan Gözükara SECourses Click to see attachment

cyberbolOP•8/15/25, 11:41 AM

How is going ?? WHen you plan publish ?? I asking as i am about start new big dataset trainig and not sure if wait for GUI or ... ???

Sakami•8/15/25, 12:09 PM

@cyberbol what GPU did you use for Wan2.2 training?

SSakami @cyberbol what GPU did you use for Wan2.2 training?

cyberbolOP•8/15/25, 1:20 PM

4090 (24 GB VRAM) – it was fine, but I’m now collecting a better dataset for my final LoRA for WAN 2.2.
For example, right now when I train WAN 2.2 with high noise and low noise… the low noise results are actually better when I use my old LoRA from WAN 2.1.
So currently I’m using WAN 2.2 high noise for the high noise pass, and my old WAN 2.1 LoRA for the low noise pass.

I’m not sure why… Most of the issues seem to be with human eyes.
I think what I need to do is:

for low noise → collect a dataset with high resolution, close-ups on skin details, eyes, etc.

for high noise → use more general shots of the body, face, etc.

cyberbolOP•8/15/25, 1:23 PM

Key Training Parameters for WAN2.2 (for RTX 4090)

For the High-Noise model:

Optimizer: adamw8bit (a key optimization for 24GB VRAM)
Precision: fp16 and fp8_base
Learning Rate: 3e-4
LR Scheduler: polynomial with power=8
Network Dim / Alpha (LoRA Size): 16 / 16
Gradient Checkpointing: Enabled

Timestep Range:
min_timestep: 875
max_timestep: 1000

For the Low-Noise model:

All parameters are identical to the High-Noise model, except for the timestep range.

Optimizer: adamw8bit
Precision: fp16 and fp8_base
Learning Rate: 3e-4 or other 2e-4
LR Scheduler: polynomial with power=8
Network Dim / Alpha (LoRA Size): 16 / 16
Gradient Checkpointing: Enabled

Timestep Range:

min_timestep: 0
max_timestep: 875

Ccyberbol How is going ?? WHen you plan publish ?? I asking as i am about start new big da...

Furkan Gözükara SECourses•8/15/25, 1:44 PM

my research would take at best several days

FFurkan Gözükara SECourses my research would take at best several days

cyberbolOP•8/15/25, 1:57 PM

until publish gui ?? ohhhh..

Ccyberbol until publish gui ?? ohhhh..

Furkan Gözükara SECourses•8/15/25, 1:57 PM

yep

Furkan Gözükara SECourses•8/15/25, 1:57 PM

i need to train and test

cyberbolOP•8/15/25, 1:58 PM

ok, so i will use just command by now

cyberbolOP•8/15/25, 2:05 PM

I'm looking for the latest info and best practices for training WAN 2.2 LoRAs, especially for character likeness on a 24GB card.

From what I've gathered so far, it seems the best dataset approach is to train the high-noise model on low-resolution videos (to learn motion) and the low-noise model on high-resolution images (to learn the fine details).

I also saw in the new Musubi-Tuner docs that you can apparently train both high and low noise LoRAs at the same time now, using the --dit (for low) and --dit_high_noise (for high) parameters in a single command. For 24GB VRAM, the key seems to be adding the --lazy_loading option to only keep the active model on the GPU.

Has anyone tried this simultaneous training method yet?

Ccyberbol I'm looking for the latest info and best practices for training WAN 2.2 LoRAs, e...

JonkoXL•8/15/25, 2:51 PM

No need for the lazy-loading as you can just train the two lora's after each other (in command line it auto continues to the 2nd command)

JJonkoXL No need for the lazy-loading as you can just train the two lora's after each oth...

cyberbolOP•8/15/25, 2:58 PM

I know. But y can also Train both same time

JonkoXL•8/15/25, 2:59 PM

It probably wont be faster, and you could decide to just turn off your pc after the 1st lora training if you decide that

cyberbolOP•8/15/25, 2:59 PM

Probably you right. I just experiment..we will know much better in few weeks probably..

JonkoXL•8/15/25, 2:59 PM

+ I may want to use less epochs or learning rate for the other model

JonkoXL•8/15/25, 3:00 PM

i noticed the high noise model gets overtrained quicker

cyberbolOP•8/15/25, 3:01 PM

@JonkoXL Do you successfully Train any high and low noise LORA and it's working together with OTHER lora keeps human eyes good quality?

JonkoXL•8/15/25, 3:04 PM

I think eyes in Wan 2.2. will still be a problem unless we get an even higher base resolution

JonkoXL•8/15/25, 3:04 PM

https://civitai.com/models/1862320/wan22-t2v-lora-cartoon-style

Since I've been deep in training with the command line, I had a few thoughts on what might be needed

Similar Threads