i am making a very big comparison
i am making a very big comparison
F
Fand comparing with our best
CNot sure there's much point in me posting my LoRA results anymore. Seems they serve better as comedy material, anyway 
AWell i dont think he made fun of you
Arather than the "easy" available alternative
Jit's a playful joke or nod on how I wanted to train a lora, and it seems a dataset just appeared 
I like your lora's, and I like your posts.. I appreciate you
I like your lora's, and I like your posts.. I appreciate you
Ai like your contribution too cyberbol
SYou gotta have more faith in yourself – be strong
SWhat's your take on buckets ? Do you notice better results without ? I use different ratios so I tend to keep them on
Kanyone trained wan 2.2 in msubai trainer , particularly in h100 80gb
if you are trained on it
how can i speed up the traning time ?
if you are trained on it
how can i speed up the traning time ?
CI've been testing several samplers and schedulers for the FLUX KREA model, and it's hard to choose just ONE that is the best. It really depends on your creative goal.
However, I think these combinations are definitely worth using:
Sampler: DPM++ 2M
Scheduler: bong_tangent (New)
Sampler: Euler
Scheduler: Beta / beta57
Sampler: dpmpp_2m or ipndm
Scheduler: Beta / beta57
I'm currently running a small test comparing a few options. When it's ready, I'll share the results. However, I'm also waiting for @Furkan Gözükara SECourses comparison, as it will surely be more comprehensive.
However, I think these combinations are definitely worth using:
Sampler: DPM++ 2M
Scheduler: bong_tangent (New)
Sampler: Euler
Scheduler: Beta / beta57
Sampler: dpmpp_2m or ipndm
Scheduler: Beta / beta57
I'm currently running a small test comparing a few options. When it's ready, I'll share the results. However, I'm also waiting for @Furkan Gözükara SECourses comparison, as it will surely be more comprehensive.
CThat said, it seems impossible to choose one single best combination, as it depends on many factors like the result you want to achieve, the style of the photo, and of course, personal taste. But tests like this are useful because we can rule out the samplers/schedulers that cause issues, like artifacts.
CSampler: DPM++ 2M
Scheduler: Karras
Sooo perfection, realistic... Like Me..
Scheduler: Karras
Sooo perfection, realistic... Like Me..
JIt took Edison 1000 tries to make the light bulb.
Ano he stole the design from tesla
JYeah that makes it even worse for Edison
JEven with Tesla's blueprints, it took him 1000 tries 
JWhat happens if you use UNI_PC with SGM_UNIFORM
Ai hate edison
CGPU will burn
CHope @Furkan Gözükara SECourses will show us big comparision
CTne we can vote which one is best combination!! Yee voting good idea
Ajust amazing
Jsame
Ai love Ada Lovelace
SDo you know if it's possible to make those new samplers/schedulers work in SwarmUI ? I believe they come from RES4LYF nodes in comfy right ?
CYes, you need use a comfyui as Beckends for SmwarUI and install this mentioned by you RES4LYF
SAwesome. Thanks mate
C
Cno way to choose lol.
C
J4th row 1st column of the blue ones on the boat
J1st row last column also
J
JAnimation I made using my Lora + a new workflow I am working on (for extending videos)
CSooo many combination..lol. I need go for a walk.. clear my brain.
PROMPT:
"close headshot portrait photograph of white ohwx man wearing a super expensive white suit, a light blue shirt and a black tie in a photo studio lighting hdr. The studio backdrop is a rich, deep black that makes the white suit pop dramatically. Professional lighting creates a perfect balance of highlights and shadows, accentuating his facial features. The suit is impeccably tailored, with subtle textures visible in the fabric. His tie is made of the finest silk, with a subtle pattern that catches the light. His hair is perfectly styled, and his expression is one of confident professionalism.<segment:yolo-face_yolov9c.pt-1,0.7,0.5>photograph of ohwx man"
EULER / BETA
30/40 steps
CFG: 1 , 2, 2.5
PROMPT:
"close headshot portrait photograph of white ohwx man wearing a super expensive white suit, a light blue shirt and a black tie in a photo studio lighting hdr. The studio backdrop is a rich, deep black that makes the white suit pop dramatically. Professional lighting creates a perfect balance of highlights and shadows, accentuating his facial features. The suit is impeccably tailored, with subtle textures visible in the fabric. His tie is made of the finest silk, with a subtle pattern that catches the light. His hair is perfectly styled, and his expression is one of confident professionalism.<segment:yolo-face_yolov9c.pt-1,0.7,0.5>photograph of ohwx man"
EULER / BETA
30/40 steps
CFG: 1 , 2, 2.5
CHey everyone,
Kicking off a new training run for my Qwen-Image likeness LoRA today. This time, I'm diving into some of the newer, more advanced parameters available in Musubi-Tuner to see if they can improve the final quality.
Here's what I'm experimenting with:
LoRA+: Using a loraplus_lr_ratio of 4, which is supposed to make the training more efficient by using a higher learning rate on part of the network.
PyTorch Dynamo: Enabled the inductor backend. I'm curious to see if the JIT compiler gives any extra speed boost on top of the performance gains from WSL.
New Timestep Sampler: Switched to the qinglong_qwen sampler. It's a hybrid method that's reportedly better for style and likeness learning.
Post-Hoc EMA Merging: After the training is done, I'll also be using the new script to merge the best checkpoints into one final model instead of just picking a single one. The goal is a more stable and accurate LoRA.
The training is running in WSL2 (Ubuntu) using the Musubi-Tuner scripts on an RTX 4090.
I'm really curious to see what effect these new parameters will have on the final result. I'll let you know how it goes!
Kicking off a new training run for my Qwen-Image likeness LoRA today. This time, I'm diving into some of the newer, more advanced parameters available in Musubi-Tuner to see if they can improve the final quality.
Here's what I'm experimenting with:
LoRA+: Using a loraplus_lr_ratio of 4, which is supposed to make the training more efficient by using a higher learning rate on part of the network.
PyTorch Dynamo: Enabled the inductor backend. I'm curious to see if the JIT compiler gives any extra speed boost on top of the performance gains from WSL.
New Timestep Sampler: Switched to the qinglong_qwen sampler. It's a hybrid method that's reportedly better for style and likeness learning.
Post-Hoc EMA Merging: After the training is done, I'll also be using the new script to merge the best checkpoints into one final model instead of just picking a single one. The goal is a more stable and accurate LoRA.
The training is running in WSL2 (Ubuntu) using the Musubi-Tuner scripts on an RTX 4090.
I'm really curious to see what effect these new parameters will have on the final result. I'll let you know how it goes!
DI got a question..in one trainer.. when setting the captions from one txt file (ohwx man) does this apply when training a acharacter.. same body chape and size..and face too... or is it only for face... my images have tags and it kind of threw me off abit..
Snice! how many seconds do you think are achievable while keeping the context
J5 seconds, since that's the 81 frames generated per clip.
I noticed that when she turned her back, and then turned to the front again (as a test) she looked different. (when reusing the last frame)
I noticed that when she turned her back, and then turned to the front again (as a test) she looked different. (when reusing the last frame)
JHopefully they will make an IP Adapter for Wan2.2
Snice
I’ve heard that some folks managed to get 7 to even 10 seconds in a single generation, but I’m not an expert in WAN so i dunno if maybe this was some diffrent case
I’ve heard that some folks managed to get 7 to even 10 seconds in a single generation, but I’m not an expert in WAN so i dunno if maybe this was some diffrent case
Jyes it can, but it may distort a bit.
any thing above a single clip length will lose context tho
any thing above a single clip length will lose context tho
Junless we get ip adapters or some kind of forging stuff
Acyber
Aso far flux krea or qwen?>
Awhat do you like
CDamn.. cant choose one. I like both. Just finished second training qwen lora with new parameters. I like result. will post soon.
CJust wanted to share an update: I've finished my new Qwen-Image LoRA training, and I'm really happy with the results! It's a huge improvement over my first attempt.
I think the new, advanced features in Musubi-Tuner made a massive difference. For this run, I used:
LoRA+ to make the training more efficient.
PyTorch Dynamo (inductor backend) to compile and optimize the model on the fly in WSL.
The qinglong_qwen sampler for a more intelligent learning process.
After the training was complete, I picked my top 3 favorite checkpoints. Then, instead of just choosing one, I merged them into a single, final model using the lora_post_hoc_ema.py script. This command averages the weights of the best LoRAs to create a more stable and higher-quality final version.
There's still some work to do on finding the best sampler and scheduler, but I'm very pleased with the outcome so far. The training took a bit longer than expected, around 4 hours on my RTX 4090, which was likely due to the initial, slow compilation phase of PyTorch Dynamo that needs time to warm up.
Here are a few example images generated with the final merged LoRA + 8stepLora.
I think the new, advanced features in Musubi-Tuner made a massive difference. For this run, I used:
LoRA+ to make the training more efficient.
PyTorch Dynamo (inductor backend) to compile and optimize the model on the fly in WSL.
The qinglong_qwen sampler for a more intelligent learning process.
After the training was complete, I picked my top 3 favorite checkpoints. Then, instead of just choosing one, I merged them into a single, final model using the lora_post_hoc_ema.py script. This command averages the weights of the best LoRAs to create a more stable and higher-quality final version.
There's still some work to do on finding the best sampler and scheduler, but I'm very pleased with the outcome so far. The training took a bit longer than expected, around 4 hours on my RTX 4090, which was likely due to the initial, slow compilation phase of PyTorch Dynamo that needs time to warm up.
Here are a few example images generated with the final merged LoRA + 8stepLora.
CDataset just 20 images like last time.
Command in Musumi-TUner to start:
python3 src/musubi_tuner/qwen_image_cache_latents.py --dataset_config dataset/dataset_qwen_test.toml --vae models/qwen_image/vae/vae/diffusion_pytorch_model.safetensors
python3 src/musubi_tuner/qwen_image_cache_text_encoder_outputs.py --dataset_config dataset/dataset_qwen_test.toml --text_encoder models/qwen_image/text_encoders/split_files/text_encoders/qwen_2.5_vl_7b.safetensors
PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True accelerate launch --num_cpu_threads_per_process 1 src/musubi_tuner/qwen_image_train_network.py \
--dataset_config dataset/dataset_qwen_test.toml \
--dit models/qwen_image/diffusion_models/split_files/diffusion_models/qwen_image_bf16.safetensors \
--text_encoder models/qwen_image/text_encoders/split_files/text_encoders/qwen_2.so_vl_7b.safetensors \
--vae models/qwen_image/vae/vae/diffusion_pytorch_model.safetensors \
--output_dir output_qwen_advanced \
--output_name Qwen-LoRA-Arek-Advanced \
--mixed_precision bf16 \
--sdpa \
--optimizer_type adamw8bit \
--learning_rate 2e-4 \
--lr_scheduler cosine_with_restarts \
--lr_warmup_steps 300 \
--lr_scheduler_num_cycles 3 \
--network_module networks.lora_qwen_image \
--network_dim 32 \
--network_alpha 16 \
--network_args "loraplus_lr_ratio=4" \
--max_train_steps 3000 \
--save_every_n_steps 250 \
--gradient_checkpointing \
--timestep_sampling qinglong_qwen \
--dynamo_backend INDUCTOR \
--fp8_base \
--fp8_scaled \
--blocks_to_swap 20
Command in Musumi-TUner to start:
python3 src/musubi_tuner/qwen_image_cache_latents.py --dataset_config dataset/dataset_qwen_test.toml --vae models/qwen_image/vae/vae/diffusion_pytorch_model.safetensors
python3 src/musubi_tuner/qwen_image_cache_text_encoder_outputs.py --dataset_config dataset/dataset_qwen_test.toml --text_encoder models/qwen_image/text_encoders/split_files/text_encoders/qwen_2.5_vl_7b.safetensors
PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True accelerate launch --num_cpu_threads_per_process 1 src/musubi_tuner/qwen_image_train_network.py \
--dataset_config dataset/dataset_qwen_test.toml \
--dit models/qwen_image/diffusion_models/split_files/diffusion_models/qwen_image_bf16.safetensors \
--text_encoder models/qwen_image/text_encoders/split_files/text_encoders/qwen_2.so_vl_7b.safetensors \
--vae models/qwen_image/vae/vae/diffusion_pytorch_model.safetensors \
--output_dir output_qwen_advanced \
--output_name Qwen-LoRA-Arek-Advanced \
--mixed_precision bf16 \
--sdpa \
--optimizer_type adamw8bit \
--learning_rate 2e-4 \
--lr_scheduler cosine_with_restarts \
--lr_warmup_steps 300 \
--lr_scheduler_num_cycles 3 \
--network_module networks.lora_qwen_image \
--network_dim 32 \
--network_alpha 16 \
--network_args "loraplus_lr_ratio=4" \
--max_train_steps 3000 \
--save_every_n_steps 250 \
--gradient_checkpointing \
--timestep_sampling qinglong_qwen \
--dynamo_backend INDUCTOR \
--fp8_base \
--fp8_scaled \
--blocks_to_swap 20
