depends i use different settings, but usually adafactor and the extra arguments with the gradient ch
depends i use different settings, but usually adafactor and the extra arguments with the gradient checkpointing and sometimes xformers not always.
LDo you think adafactor is better than AdamW (not the 8 bit)
FFor me for faster training with lora i always use adamw8bit, dreambooth still testing but best is adafactor
Fand for sdxl i use the noise ofset 0.0357
Falso using a older version of kohya uses less vram i think 21.8.4
Falso if you want me to train your data set on my system i can. so we canm compare training times
LI’m just playing around at the moment I’m going to look at the items you mentioned. What about the basic settings like images, epochs, repeats? What I’m struggling with real people. I followed the doctors videos and did the first 2 and got just ok results.
FSo far people, I use 15-30 Images, no harsh angles , usually front, 3/4 and side shots.
20 Repeats seems to be the golden amount.
Also i found using too much class images affects my training,
and usually 8 epochs, I generate samples as well, so I usually stop the training early.
20 Repeats seems to be the golden amount.
Also i found using too much class images affects my training,
and usually 8 epochs, I generate samples as well, so I usually stop the training early.
FAlso you may want to try using min snr gamma at the value of 10
LHow many class images roughly?
FI have my own set of 2000, but it only uses around 500
the more repeats you have the more class images it uses
the more repeats you have the more class images it uses
LAnd what SDXL base model do you use?
FIts a bit tricky so far, i use the base sdxl model 1.0, But usually loras dont work too well with other checkpoints.
So im using a custom merge of base xl juggernaught xl dreamshaper and realvis,
So im using a custom merge of base xl juggernaught xl dreamshaper and realvis,
Fbut if you're only use base xl, then yeah base xl is the best
Fwill probably make my own tutorial soon, still testing for best settings
LI’m setting this up that the moment thanks so much for the help. Learning rates?
FAre you training lora?
LYes
FIf you're using adafactor
0.0001
0.0001
0.0001
for all 3
0.0001
0.0001
0.0001
for all 3
FExtra arguments
scale_parameter=False relative_step=False warmup_init=False
scale_parameter=False relative_step=False warmup_init=False
Fyou can also try one with weight decay
scale_parameter=False relative_step=False warmup_init=False weight_decay=0.01
scale_parameter=False relative_step=False warmup_init=False weight_decay=0.01
LOk I think I have almost everything. Question though. The Doctor proves 5200 class images. Do I use the whole folder or pull some out.
Fhmmm i havent used his ones, cause some of the images there you don't want to use.
So i'd just pull out the best ones from there, take about 1000>
get nice clear shot images
So i'd just pull out the best ones from there, take about 1000>
get nice clear shot images
Yehm, hey everyone, does anyone have experience with kohya ss?
LAwesome, what should the network Dimm and Alpha be?
FTheres a few you can try
128 32
96 32
64 32
i mainly go with 128 32
128 32
96 32
64 32
i mainly go with 128 32
FYes i know some videos and tutorials say to use alpha of 1, but it takes longer to converge and sometimes doesnt work to well
FYep whats up
FAlso if you feel like constant learning rate is cooking your image, switch to cosine
YI'm trying to see if its possible to just pass in the link of a huggingface dataset instead of making an image folder and linking it with captions
Yespecially on the gui side
LYou know what, I’ll bet that’s the biggest issue on time. Been using 1 on Alpha
FIt shouldnt be hmmm, are you usingMemory efficient attention >
LI don’t think so
Fare you training locally or runpod
Yon google colab
Fhow about gaming? noticing any issues with the 4090?
Fahh never used google collab, you should try kaggle or runpod
LNot a bit, crushes Baldur’s 3 on max
Ythey should work the same I think
Ydid work with kaggle before and it's pretty similar
Fyou should be good then, if training is still slow lemme know
Falso do you have harware-acclerated gpu scheduling turned on?
Fdamn might have to wait till somene gets back to you 
idk google collab
idk google collabLDo you change the LR scheduler?
LAnd B16 or F16?
Yah okay, honestly I'm only looking at this
FYep metioned it above ^ Also if you feel like constant learning rate is cooking your image, switch to cosine
Fbf16 since you're on a 4090
Ftry putting it like this /stabilityai/stable-diffusion-xl-base-1.0
Yah, that's chill, I think it wants me to download it and give it the full path. I can do that