you mean when it goes into training the toml?
you mean when it goes into training the toml?
Fin saved json file
Fi made a fresh one 
Mwhatever was in the json gets trained on the previous kohya one, the latest it doesnt
Mlemme check patreon
Fi am doing new tests
Fclicp skip discerte value
Mwhich post did you update it on?
Fi didnt update again v21 is latest
Fi will update tomorrow hopefully
Moh no worries i will check tomorrow 
Fi will test clip skip 2 as well
Malso xformers seems to be worksing
Mworking*
Fi didnt see its benefit in terms of speed or vram so didnt bother
Mseems faster for me than sdpa
Mand 1gb less vram
Fi should test on windows
Fi tested on ubuntu :d
Fquality also matters
Mohhh maybe windows with the new torch
Mtesting quality now
Mfirst try on xformers
Fwhich xformers you installed
Mlet me check for you
Fi got xformers==0.0.27.post2+cu118
Fgonna train on this
Mjust checked
Myeah using same one
Mpip install xformers==0.0.27.post2
Fye uses lesser vram and looks alitt bit fater
Fif same quality i will update all configs :d
Fok vram usage became same :d
F
Mi think for linux xformers works fine because of the torch version
Mbut for windows i think its using none, no xformers or sdpa ?
MMi get this here on widnows
Mbut its much faster for me than sdpa
Minteresting
GI am unsure how epoch works. I see your settings always has 200, why change it? Is 1000 images too much to train a style?
Why touch or change epochs? Thats how steps are determined?
Why touch or change epochs? Thats how steps are determined?
GCan I get that coupon and website please? @Dr. Furkan Gözükara
DFrom what I understand (correct me, if I am wrong, Doc), an epoch equals how many times an image in the input dataset is trained.
So suppose you have an input dataset of 15 images, and you select an epoch of 200, each image will be trained 200 times, so the No. of steps = 15x200 = 3000 (3000 steps is a trade-off figure b/w time and quality that the Doc likes to use in his videos).
Now, the time taken for each step will depend upon your GPU. So, say a 12GB GPU trains with Flux at 12 second/ iteration. Then, the total time =
3000 x 12 = 36,000 secs (approx. 10hrs.)
So suppose you have an input dataset of 15 images, and you select an epoch of 200, each image will be trained 200 times, so the No. of steps = 15x200 = 3000 (3000 steps is a trade-off figure b/w time and quality that the Doc likes to use in his videos).
Now, the time taken for each step will depend upon your GPU. So, say a 12GB GPU trains with Flux at 12 second/ iteration. Then, the total time =
3000 x 12 = 36,000 secs (approx. 10hrs.)
DThe above is when you use repeat = 1,
So no. of steps = epoch x repeats x no. of images in input dataset
So no. of steps = epoch x repeats x no. of images in input dataset
DAnd time taken (in hrs.) = (no. of steps x time taken for each iteration by the GPU)/3600
DWhat I do not understand (yet!), however, is:
1) what difference it makes if you use 3000 steps versus, say 7000 steps (increase the epoch for the same no. of input images. My assumption is it takes more time, but the quality will be better to a certain extent, before it gives diminishing returns and then peaks).
2) Use less images & more epoch, versus more images and less epoch (However, the no. of steps is maintained the same. Eg. 15x200=3000, and 50x60=also3000. Here, I make no assumption and would like to hear from Doc).
1) what difference it makes if you use 3000 steps versus, say 7000 steps (increase the epoch for the same no. of input images. My assumption is it takes more time, but the quality will be better to a certain extent, before it gives diminishing returns and then peaks).
2) Use less images & more epoch, versus more images and less epoch (However, the no. of steps is maintained the same. Eg. 15x200=3000, and 50x60=also3000. Here, I make no assumption and would like to hear from Doc).
D1) largely correct, but there is no peak, there is a cliff. you can overfit diffusion models quite easily
D2) there is no clear answer, it's an art. Less images tend to give you an inflexible result, more images tend to be harder to train until the model picks up the details. You often need more steps in total for many images than for a few, meaning not reduce the epochs as much as you have increased the number of images. Larger batch sizes can help with (many) more images. Adjustments in LR might be necessary.
Dand to make it even less predictable, it depends on the dataset, especially how similar the images are
DYes, think I understand.
Good to see I got the basics right.
Good to see I got the basics right.
DThanks for explaining further.