training on layer 7 and layer 20 is not too bad. Using noise_scheduler: flowmatch, optimizer: adamw
training on layer 7 and layer 20 is not too bad. Using noise_scheduler: flowmatch, optimizer: adamw8bit and lr: 0.0005
Fye your LR is 10x of mine 
Fi am doing more trainings
Fatm
Z@Dr. Furkan Gözükara so supir tool was built using cudatoolkit 11.8? but xformers is looking for pytorch 2.4.0 +cu121, i'm confused
Znot sure why it says that cause i don't even have pytorch 2.2.0 installed i have 2.4.0
PQuestion in the round of you awesome nerds 
. Iw as trying the last days control net with flux in SD Forge webUi but it seems broken.. neither the Depoth or the Canny mode worked for me .. Any one having the same issue ?
. Iw as trying the last days control net with flux in SD Forge webUi but it seems broken.. neither the Depoth or the Canny mode worked for me .. Any one having the same issue ?
Fuse our installer it handles all
SWill you be doing finetune vs lora comparisons?
Fyep i already did too
SOh you already did?
Fcheck images
FPatreon
Get more from SECourses: Tutorials, Guides, Resources, Training, FLUX, MidJourney, Voice Clone, TTS, ChatGPT, GPT, LLM, Scripts by Furkan Gözü on Patreon
Fleft best lora right fine tune
SHmmm I see, finetune better for sure. Have you also tried finetune lora extract? I tried that but the quality dropped drastically, something was definitely wrong with the lora extract script
Fnot tried yet
Fbut it should be better than lora training it self
SYeah I also think that. I'd use it but that lora extract script is fully broken, atleast for me. And keeping those huge checkpoints is a bit much
�what network dim and alpha are you using?
K@Dr. Furkan Gözükara I can't seem to find that captioning editor GUI you've made. Would you mind linking to it?
DI want to try this but I have some questions, layes=bloks what's the diference between single and double blocks? you are testing with single blocks or layers? can I test with 7 and 20 using a higher lr, let us know your findings, thanks
F128x128
FPatreon
Get more from SECourses: Tutorials, Guides, Resources, Training, FLUX, MidJourney, Voice Clone, TTS, ChatGPT, GPT, LLM, Scripts by Furkan Gözü on Patreon
Fthat is what i am testing now :d
Fdone 16 trainings so far training 8 more
�You can try 256 if you don't see good results
Fyes can be improved nice idea to test
Dlooks like individual block training might have been implemented in the sd3 flux1 kohya ss branch last night
XSo fine tuning is better the Lora over all you think @Dr. Furkan Gözükara ?
D
Fi am testing already
XYou have a video on how to fine tune training?
RCan someone confirm Flux Lora Extraction in completely broken on Kohya ?
RI did a finetune of flux dev, results are amazing. I use Extract Flux LoRA tab to extract the Lora from the finetuned model. When I use the extracted Lora it makes no difference with not using Lora. @Dr. Furkan Gözükara Did you try the Extract Flux LoRA feature ?
XI’m confused on what fine tuning is, do we basically do the same training as in this video https://youtu.be/nySGu12Y05k?feature=shared but with the configuration from the new update on your patreon?
YouTubeSECourses
Ultimate Kohya GUI FLUX LoRA training tutorial. This tutorial is product of non-stop 9 days research and training. I have trained over 73 FLUX LoRA models and analyzed all to prepare this tutorial video. The research still going on and hopefully the results will be significantly improved and latest configs and findings will be shared. Please wat...
NI'd also like to know. Though I was waiting on Kohya to refine it a bit more before delving into fine tunes.
SSame here. Do you mean extracting a lora from a finetuned checkpoint?
NYeah
SFor me the extracted lora looks nothing like my subject, nothing like the checkpoint
SI guess it is broken
R@smoke yeah, thx for the confirmation
SShould probably raise an issue in sd-scripts Github since it is broken
RGitHub
2 Issues sometimes it works, but it looks nothing like the fine tuned model. Doesnt work and i get the error below During handling of the above exception, another exception occurred: Traceback (mos...
SAh yeah it definitely is broken
RBut i'm not sure it's the same issue
RBecause they say it works at 1.5 strength
RBut for me at 2.0 there's was no difference
SWell for me it is, the extracted lora does not look like my finetuned checkpoint. At 1.5 strength it starts to look more like it but it created a lot of other issues and graininess, so definitely not the best approach
SAtleast for me
RComfy ?