i think for linux xformers works fine because of the torch version
i think for linux xformers works fine because of the torch version
Mbut for windows i think its using none, no xformers or sdpa ?
MMi get this here on widnows
Mbut its much faster for me than sdpa
Minteresting
GI am unsure how epoch works. I see your settings always has 200, why change it? Is 1000 images too much to train a style?
Why touch or change epochs? Thats how steps are determined?
Why touch or change epochs? Thats how steps are determined?
GCan I get that coupon and website please? @Dr. Furkan Gözükara
DFrom what I understand (correct me, if I am wrong, Doc), an epoch equals how many times an image in the input dataset is trained.
So suppose you have an input dataset of 15 images, and you select an epoch of 200, each image will be trained 200 times, so the No. of steps = 15x200 = 3000 (3000 steps is a trade-off figure b/w time and quality that the Doc likes to use in his videos).
Now, the time taken for each step will depend upon your GPU. So, say a 12GB GPU trains with Flux at 12 second/ iteration. Then, the total time =
3000 x 12 = 36,000 secs (approx. 10hrs.)
So suppose you have an input dataset of 15 images, and you select an epoch of 200, each image will be trained 200 times, so the No. of steps = 15x200 = 3000 (3000 steps is a trade-off figure b/w time and quality that the Doc likes to use in his videos).
Now, the time taken for each step will depend upon your GPU. So, say a 12GB GPU trains with Flux at 12 second/ iteration. Then, the total time =
3000 x 12 = 36,000 secs (approx. 10hrs.)
DThe above is when you use repeat = 1,
So no. of steps = epoch x repeats x no. of images in input dataset
So no. of steps = epoch x repeats x no. of images in input dataset
DAnd time taken (in hrs.) = (no. of steps x time taken for each iteration by the GPU)/3600
DWhat I do not understand (yet!), however, is:
1) what difference it makes if you use 3000 steps versus, say 7000 steps (increase the epoch for the same no. of input images. My assumption is it takes more time, but the quality will be better to a certain extent, before it gives diminishing returns and then peaks).
2) Use less images & more epoch, versus more images and less epoch (However, the no. of steps is maintained the same. Eg. 15x200=3000, and 50x60=also3000. Here, I make no assumption and would like to hear from Doc).
1) what difference it makes if you use 3000 steps versus, say 7000 steps (increase the epoch for the same no. of input images. My assumption is it takes more time, but the quality will be better to a certain extent, before it gives diminishing returns and then peaks).
2) Use less images & more epoch, versus more images and less epoch (However, the no. of steps is maintained the same. Eg. 15x200=3000, and 50x60=also3000. Here, I make no assumption and would like to hear from Doc).
D1) largely correct, but there is no peak, there is a cliff. you can overfit diffusion models quite easily
D2) there is no clear answer, it's an art. Less images tend to give you an inflexible result, more images tend to be harder to train until the model picks up the details. You often need more steps in total for many images than for a few, meaning not reduce the epochs as much as you have increased the number of images. Larger batch sizes can help with (many) more images. Adjustments in LR might be necessary.
Dand to make it even less predictable, it depends on the dataset, especially how similar the images are
DYes, think I understand.
Good to see I got the basics right.
Good to see I got the basics right.
DThanks for explaining further.
Nit writes errors, do I have to put in an experimental file?
THey, does anyone have a prodigy config for flux? Back in the day we’d set the warmup steps to 100% and have loras ready in 30-45 mins or so depending on the amount of images in the dataset.
I like the results that your rank 3/4 config provides, it’s really high quality. But I’d need something a bit faster than 35 hrs for a 300 image dataset - any tips would be really appreciated
I like the results that your rank 3/4 config provides, it’s really high quality. But I’d need something a bit faster than 35 hrs for a 300 image dataset - any tips would be really appreciated
Fyou were right i see improvment in the training
Flooks like bmaltais fixed something for sure
Fprev vs new
Fclothing much better definitely same settings
TWhat do we need to do to update? Just run the step 2 bat?
Fposted after bug fix training results : https://www.reddit.com/r/StableDiffusion/comments/1f6b948/kohya_gui_fixed_a_bug_yesterday_and_after/
Reddit
Explore this post and more from the StableDiffusion community
Fand retrain of course
Fbut you can wait me to update configs as well
Fbecause i am gonna add text encoder training
Fwhich certainly improves quality
Fcoupon is SECourses
Fselect our image
GWhats the website?
GAnd since you are here. I want to train 1000 NFTs to grab the art style that I made. Is that too many? I see on my a6000 it would take like 450 hrs!
Flet me show you
Gits 1000 too many to train a style?
FYouTubeSECourses
In this video, I demonstrate how to install and use #SwarmUI on cloud services. If you lack a powerful GPU or wish to harness more GPU power, this video is essential. You'll learn how to install and utilize SwarmUI, one of the most powerful Generative AI interfaces, on Massed Compute, RunPod, and Kaggle (which offers free dual T4 GPU access for ...
Ffirst 10 minute
Fto learn massed compute
Gits Kohya too right
Fsite link
FFit has kohya
Fi made a tutorial for kohya
Fbut still editing
Fyou can wait it too
Fif you dont want to struggle
Gis 1000 too many images for training a style?
Fnope better
Fas long as dataset is good
Ghow about 2000?
