for lora i recommend multiple gpu
for lora i recommend multiple gpu
Fto speed up
Ffor lora i dont have batch config but you can use batch size 1
Fand set batch size 7
Fand multiply learnign rate with
Fsquare root 7 * base learning rate
Fhowever batch size 1 has the best quality
SI am using your Kaggle Kohya ss file to start a kohya ss gradio ui, but it fails by saying no conda environment found, and if i manually start i and try to load a json file forf the configuration, it says something like 'str' has not attribute 'get'? please help
S
Fwhich versioın are you using?
Ffile name
Si guess its v25, i just ran it using the py file let me try
Sits 2 am now, i will let you know later what happens
Flatest is v28
Fuse it
FPatreon
Get more from SECourses: FLUX, Tutorials, Guides, Resources, Training, Scripts on Patreon
MIs it even possible to set batch size 7 with VRAM 48GB? Because even with batch size 1, VRAM usage is around 36GB
MFYI, I want best quality, but I should also consider training speed
Fi think not with lora
Fit is for dreambooth at 1024x1024
MOh, I see
MFor dreambooth, batch size 1 & 7 gpu vs batch size 7 & 1 gpu
MIs this same speed?
MI assume quality would be better with batch size 1
Fof course not
Fnormally 7 gpu means it scales
Fwith lora almost 7x speed
Fbut for dreambooth you need 80 gb gpus and i didnt test
Fbatch size 7 is like 1.3x speed
MThen, best way to speed up the training is
- lora
- batch size 1
- multiple gpus(for linear speed up)
MAdditional question, is there any way that I can improve training speed itself?
- I used best Tier 1 config for 48GB vram for lora training
- But, set "cache_latents = false / cache_latents_to_disk = false"
- With RTX 6000 Ada, it took 24 mins for 20 epoch
- With A100 SXM, it took 16 mins for 20 epoch
Fsadly not much else
Fgraident checkpointing slows down
Fbut turning it of requires huge vram
Mhmmm... in that case, I need to think about how to optimize gpu cost
MMI'm curious how much quality difference between fp8 and fp16
Fnot possible
Feven if you use fp8 weights the training happens at 16 bit precision
Fkohya doesnt support fp8 training
MThank you so much for answering all the questions! I'll focus on facial accuracy with Kohya fisrt, and then think about speed later haha
ZI see, thank you.
Where we can see the class type for this flux model?
Where we can see the class type for this flux model?
Fthere is no class types
Fyou need to understand yourself sadly
HYoo devs
U@Furkan Gözükara SECourses Hello 
I was wondering if you have any plans in the future to train a custom lora for wan 2.1/2.2 (when it releases) video generation?
I was wondering if you have any plans in the future to train a custom lora for wan 2.1/2.2 (when it releases) video generation?
Fi saw today wan 2.2 coming haha
UCant wait for your video 
UWhen should I be looking out for it?