i dont think they are trainable but didnt test
i dont think they are trainable but didnt test
Vyou should upgrade to torch 2.4.0 and vision 19.0
Vit uses the new cudnn 9.01
Vi think faster
Vtorch==2.4.0+cu124 --index-url https://download.pytorch.org/whl/cu124
torchvision==0.19.0+cu124 --index-url https://download.pytorch.org/whl/cu124
torchvision==0.19.0+cu124 --index-url https://download.pytorch.org/whl/cu124
Vhavent seen training on it yet, so not sure
Flet me test these to see vram usage
Vapprently 50% to 100% speed boost
Vbut i think only for fp16 and bf16
Fnice
Fi am gonna test later
Fi dont want to take risk of breaking quality :d
Vi havent noticed quality difference 
but you can try later
but you can try laterFdid you see speed diff?
Fconfigs do make huge difference
Fso many people blindly using and not testing 
Vi think so, i am on 4090 and it feels much faster, i will try some more tests and compare speeds for you
K@Dr. Furkan Gözükara are you planning on making joy caption ?
Fyep
Fon my next list
Fhopefully today or tomorrow
Fplease remind me again tomorrow
![Dazzastrous [4090]](https://cdn.discordapp.com/avatars/871389071686135838/c4a5f4d81d9eeb18d856295d692e2c07.webp?size=40)
Dyeh its really good
Fi will add batch caption too and cloud as well
KThank you 
I will subscribe Patreon . Saw your video on YouTube
I will subscribe Patreon . Saw your video on YouTubeFawesome ty
Ddid ur dataset include pics of you not wearing specs?
![Dazzastrous [4090]](https://cdn.discordapp.com/avatars/871389071686135838/c4a5f4d81d9eeb18d856295d692e2c07.webp?size=16)
�I really think it has something to do with the thin frames that are almost invisible around the lenses and that how the sides are black too and blend in with his hair color but I dunno I'm no expert
Fdataset
Fflux lora training with 9 gb
Fdifferent expression testing - dataset has none
D9 gb lora training?!?!?!
Fyep
Fwith 128 rank it uses 9.8 gb
Fi am updating patreon
DI cant wait
DAs always you rock!
Fconfig added
Fi dont have the results yet will have tomorrow hopefully
Fset according to my best config so far
V@Dr. Furkan Gözükara 1.16s/it for me
F24 second / it for RTX 3060
V17gb vram
Fsuper fast
Fon A6000 it is 8.3 second / it but didnt test latest
Fwill do today
Vyou should get 50% speed boost on your 3090
F17 gb is also amazing