i am testing lora extract right now out of flux fine tune
i am testing lora extract right now out of flux fine tune
F
Fextracting right now
Ftesting result
Fye quality of lora reduced a lot
Fdoing another test
Ffine tune vs lora
F
RThat's better than my lora, it has no influence even at 2.0 strength
Fthis is str 1
Rclassic extractor have a Minimum difference you could set to 0.0001. That made all the difference when you extract.
Fit was for text encoder
Ftesting 0.01 clamp
Fclamp broken it
Fhere opened an issue
FGitHub
I have used below config to extract 128 LoRA Network Rank with 0.99 Clamp Quantile and results are really bad What does Clamp Quantile do @kohya-ss ? Then I tried 0.01 Clamp Quantile and it is all ...
RThank you for looking at the issue
Fok clamp fixed
F
Fclamp 1 almost same
Fnow will try 256 rank
Factually extracted lora works even better haha 
F256 rank
Fok testing 512 :d
F512 rank
F
Ftesting 1024
F1024 failed
Fi will do a grid post of all and post on patreon
RClamp 1 definitely helped, I also had to use a checkpoint that I considered overfitted as a finetuned model form but increased the likeness in the Lora
Ftesting 768
Fye it makes huge diff
Fwow extraction uses so minimal amount of vram
F3800 mb
Ftesting fp16 vs float vs bf16 saving
Fextracted lora is lesser overfit
F
Fbut there is a little bit resemblance lost, perhaps a more overfit fine tuned model can be used
For perhaps need bigger lora str will test
RYeah overfit fine tune definitly helped
Fi am about to complete grids
Ftesting lora weights too
F1.1 is almost identical
Fye lora extract better than lora training
F
F
Ffine tun vs lora extract vs lora
Mare you using with weight 1.1?
DIf you are using Hires.Fix with V5 Lightning, then use my recommended settings for Hires.Fix (3 Sampling Steps, Denoising strength: 0.5 and CFG Sca...
DCompetition for Flux anymore?
DThey look about the same tbh 