for fine tuning yes
for fine tuning yes
Fuse your latest checkpoint
A'block swaps' - I presume that's somewhere in the kohya_ss gui, specifically "double_blocks_to_swap = 5 (in the .toml file)?
AI kept my VRAM right on 23.9 GB using the "Quality_1_23100MB_14_12_Second_IT.json" config (on a 24 GB card). I had to try a few tricks to reduce VRAM usage, such as switching off hardware accelerated GPU scheduling, and selecting 'Adjust for best performance' under Visual Effects in the Control Panel for Advanced System Settings. This helped a little though I still had 400 MB shared on RAM.
Fyou can also compare swap 6 7 speed difference
Fand use fastest one
Fall quality same
AI had tried 4e-06 learning rate, on 26 input images, but all of my epoch results looked similar (I was saving checkpoints every tenth epoch), so I tried again on 5e-06 LR and saw some improvement, though very gradual. I also noticed, right from my earliest checkpoint save, that sample photographic images of a 'woman' looked identical to those for 'ohxw woman', whereas for SDXL dreambooth the two images converged once the model was being over-trained in the new subject.
Fit is true that 05 06 or 03 may work better depending on dataset
Freally hard to set one LR for all
Fi tried to set an LR that will not very undertrain or overtrain
Fbut with my LR as you do more training, you should eventually get your perfect checkpoint
AYes, I eventually got a good result.
WCan you describe more out which step to do it. Where I can load the latest checkpoint and how to calculate the remain step for the latest checkpoint to continue
DI've seen Flux not really converge on realism when the LR is too low, no matter how long you run it. It changes facial features but remains in its plastic AI style with a low LR
Dare the e-06 values above with rank 128 alpha 16?
Fit wont have those
Fyou need to memorize and do yourself
Fdont change alpha we have it affects LR
DI have it at 1 but different LRs, that's why I'm asking. Alpha 16 multiplies the LR by 16 compared to 1
Frank 16 + alpha 16 you trying?
Fyou should make them equal
For your LR need to be researched again
Drank 128 alpha 1
Dthey don't have to be equal
Dalpha is there so you can change the rank without changing the LR
Fthen you need new LR
Dso whatever alpha you use, don't change it. but it doesn't matter what alpha you use.
Factually we make alpha equal to rank
Fso that you can use same LR
Dit's the other way around. if you use alpha 16, you can change the rank without changing the LR. OT (and I think kohya) automatically scale the LR by (alpha / rank)
Fi recently researched this i am 100% sure
Djust keep alpha the same
Fi even talked this with kohya
Falpha is just a multiplier that is used to balance LR for different lora ranks
Dexactly. we mean the same thing 
Ffor example when i moved to single layer, it required 10x more LR
Dkeep your alpha unchanged, keep your LR unchanged, change the rank. it'll work, because LR is automatically scaled.
Fnope opposite
D
D
Dif you have a LR that works at rank 16 alpha 16, and then change to rank 64 alpha 64, it'll fail.
Dif you change to rank 64 alpha 16 it'll still work
Dbecause it automatically lowers your LR for you. rank 64 needs a lower LR internally than rank 16
Dthis is OT. I hope kohya works the same. that would be very confusing otherwise
Fi will test this because i am pretty sure lots of inaccurate info there are
lots of inaccurate info there are
DI trained my dog model. However, I don't know what to put in the prompt to get it to inpaint or generate the photo of my dog.
Ftell me your folder name
Ftraining images
