drop your prior weight loss to 0.10
drop your prior weight loss to 0.10
Vand more steps
Fnice idea should test
Vreg images not reccomended for flux, but 0.10 and lower prior loss and higher lr ideal
Fjust testing atm. so far got great results and getting even better with finding better params
Vsame as well, been getting amazing results with finetune/dreambooth as well
Vlora is sort of crap because of the different alphas and conv needed
Ffully distant shot 
Fi like fine tuning too. but first started lora and sadly it doesnt work with 12 gb config
Fit uses 18 gb
VMafia boss 
Fi tell this to kohya he says working . i have shown full config toml everything. tested on windows and ubuntu
Fuses 18 gb with half model split training
Vwhen training on the fp8 model probably less vram
Vwhen available
Fit is fp8 plus half model
Fstill uses 18 gb
Vare you training on the full 23gb model?
FFi tested both fp8 model and fp16 model but it doesnt matter the precision i set is fp8 mode
Fthe vram usage is very erratic as shown above
Vhopefully one trainer starts working on it
Vhave your tried the bitsandbytes flux model?
Fwhat you mean by that
Faccelerator has fp8 mode but it doesnt work with kohya :d
Fi dont think they are trainable but didnt test
Vyou should upgrade to torch 2.4.0 and vision 19.0
Vit uses the new cudnn 9.01
Vi think faster
Vtorch==2.4.0+cu124 --index-url https://download.pytorch.org/whl/cu124
torchvision==0.19.0+cu124 --index-url https://download.pytorch.org/whl/cu124
torchvision==0.19.0+cu124 --index-url https://download.pytorch.org/whl/cu124
Vhavent seen training on it yet, so not sure
Flet me test these to see vram usage
Vapprently 50% to 100% speed boost
VVbut i think only for fp16 and bf16
Fnice
Fi am gonna test later
Fi dont want to take risk of breaking quality :d
Vi havent noticed quality difference 
but you can try later
but you can try laterFdid you see speed diff?
Fconfigs do make huge difference
Fso many people blindly using and not testing
Vi think so, i am on 4090 and it feels much faster, i will try some more tests and compare speeds for you
K@Dr. Furkan Gözükara are you planning on making joy caption ?
Fon my next list
Fhopefully today or tomorrow
Fplease remind me again tomorrow
![Dazzastrous [4090]](https://cdn.discordapp.com/avatars/871389071686135838/c4a5f4d81d9eeb18d856295d692e2c07.webp?size=40)
Dyeh its really good
