Memory efficient attention?
Memory efficient attention?
Fnot enabled
Fi tested it makes 0 diff
Fshared video of 9 gb training : https://www.reddit.com/r/StableDiffusion/comments/1ey6hss/kohya_ss_gui_flux_lora_training_on_rtx_3060_lora/
Reddit
Explore this post and more from the StableDiffusion community
Vinteresting, would it work with dreambooth?
Valso are you training 1024?
F1024x1024
Fdreambooth wont be this much this is half model
Vhave you noticed any difference with 512 compared to 1024?
Fnever tested 512 
Vfor me same results and 3x faster
Fsame results doesnt make sense weird :
Fi should test today
Vactually 1024 might be worse
Vflux loves 512
Vbucketing though
Vi have images that ar 3000pixels
Vand results are great
Fwhat do you mean
Vlike i train very high quality images
Vthey bucket down to 512
Vand results are amazing
Fso they are basically 512
Fdownscaled to
Vyes, but i also tested with 1024, and 512 is much better
Fi am gonna test thanks and weird :d
Vvery weird yes
Vbut flux for some reason loves 512 better
Vall good
Valso have you tried clip skip none vs clips skip 1?
Fnot yet
Fstill testing other params :d
Vtry clip skip 0 when you get a chance, i don't like kohya clip skip 1
Feach training is like 7 hours
V
Vvery long mine are usually less than 1 hour
FFi am updating versions on the server now
FTo not wait an entire day I am going to train on 2x RTX 4080 , 1024x1024 and 512x512 resolutions
Results will be in 8 - 9 hours hopefully published - got to sleep
Results will be in 8 - 9 hours hopefully published - got to sleep
Fdmn huge vram drop and speed increase with newest driver
F
Ffrom 44 gb to 27 staggering
Ffrom 8.4 second to 4.4 second
Ftesting 24 gb config
Fhuge news : https://www.reddit.com/r/StableDiffusion/comments/1ey7nwy/flux_lora_rank_128_kohya_ss_gui_training_for_8bit/
Reddit
Explore this post and more from the StableDiffusion community
Aupvoted
Fthanks. testing right now
F512px uses only 2gb lesser vram
Fbut 2x faster
KDid you update driver manually on massed compute?
Fyep