when training on the fp8 model probably less vram
when training on the fp8 model probably less vram
Vwhen available
Fit is fp8 plus half model
Fstill uses 18 gb
Vare you training on the full 23gb model?
FFi tested both fp8 model and fp16 model but it doesnt matter the precision i set is fp8 mode
Fthe vram usage is very erratic as shown above
Vhopefully one trainer starts working on it
Vhave your tried the bitsandbytes flux model?
Fwhat you mean by that
Faccelerator has fp8 mode but it doesnt work with kohya :d
Fi dont think they are trainable but didnt test
Vyou should upgrade to torch 2.4.0 and vision 19.0
Vit uses the new cudnn 9.01
Vi think faster
Vtorch==2.4.0+cu124 --index-url https://download.pytorch.org/whl/cu124
torchvision==0.19.0+cu124 --index-url https://download.pytorch.org/whl/cu124
torchvision==0.19.0+cu124 --index-url https://download.pytorch.org/whl/cu124
Vhavent seen training on it yet, so not sure
Flet me test these to see vram usage
Vapprently 50% to 100% speed boost
VVbut i think only for fp16 and bf16
Fnice
Fi am gonna test later
Fi dont want to take risk of breaking quality :d
Vi havent noticed quality difference 
but you can try later
but you can try laterFdid you see speed diff?
Fconfigs do make huge difference
Fso many people blindly using and not testing 
Vi think so, i am on 4090 and it feels much faster, i will try some more tests and compare speeds for you
K@Dr. Furkan Gözükara are you planning on making joy caption ?
Fon my next list
Fhopefully today or tomorrow
Fplease remind me again tomorrow
![Dazzastrous [4090]](https://cdn.discordapp.com/avatars/871389071686135838/c4a5f4d81d9eeb18d856295d692e2c07.webp?size=40)
Dyeh its really good
Fi will add batch caption too and cloud as well
KThank you 
I will subscribe Patreon . Saw your video on YouTube
I will subscribe Patreon . Saw your video on YouTubeFawesome ty
Ddid ur dataset include pics of you not wearing specs?
![Dazzastrous [4090]](https://cdn.discordapp.com/avatars/871389071686135838/c4a5f4d81d9eeb18d856295d692e2c07.webp?size=16)
�I really think it has something to do with the thin frames that are almost invisible around the lenses and that how the sides are black too and blend in with his hair color but I dunno I'm no expert
Fdataset
Fflux lora training with 9 gb
Fdifferent expression testing - dataset has none
D9 gb lora training?!?!?!
Fyep
Fwith 128 rank it uses 9.8 gb
Fi am updating patreon
DI cant wait