classic extractor have a Minimum difference you could set to 0.0001. That made all the difference wh
classic extractor have a Minimum difference you could set to 0.0001. That made all the difference when you extract.
Fit was for text encoder
Ftesting 0.01 clamp
Fclamp broken it
Fhere opened an issue
FGitHub
I have used below config to extract 128 LoRA Network Rank with 0.99 Clamp Quantile and results are really bad What does Clamp Quantile do @kohya-ss ? Then I tried 0.01 Clamp Quantile and it is all ...
RThank you for looking at the issue
Fok clamp fixed
F
Fclamp 1 almost same
Fnow will try 256 rank
Factually extracted lora works even better haha 
F256 rank
Fok testing 512 :d
F512 rank
F
Ftesting 1024
F1024 failed
Fi will do a grid post of all and post on patreon
RClamp 1 definitely helped, I also had to use a checkpoint that I considered overfitted as a finetuned model form but increased the likeness in the Lora
Ftesting 768
Fye it makes huge diff
Fwow extraction uses so minimal amount of vram
F3800 mb
Ftesting fp16 vs float vs bf16 saving
Fextracted lora is lesser overfit
F
Fbut there is a little bit resemblance lost, perhaps a more overfit fine tuned model can be used
For perhaps need bigger lora str will test
RYeah overfit fine tune definitly helped
Fi am about to complete grids
Ftesting lora weights too
F1.1 is almost identical
Fye lora extract better than lora training
F
F
Ffine tun vs lora extract vs lora
Mare you using with weight 1.1?
DIf you are using Hires.Fix with V5 Lightning, then use my recommended settings for Hires.Fix (3 Sampling Steps, Denoising strength: 0.5 and CFG Sca...
DCompetition for Flux anymore?
DThey look about the same tbh 
KThen it's wrong
KSo one idea maybe can be to finetune a checkpoint on a high rank and then extract lower ranks to get variated loras? 
SIs this finetune vs trained lora or extracted lora?
KIt says fine tune vs lora
KHe writes fine tune vs extracted lora vs lora on another image, so I'm guessing it's just lora
KDoes this training mean I could have ran it longer?!
SWould be strange though since all the other lora showcases it looked pretty much like him but in this comparison his lora looked nothing like him. Makes me wonder if the other ones are cherry picked
KInteresting thought
KKYou are correct... If that's a LORA it's not very good and his usually are