That's incredible! Looking forward to know more.
That's incredible!
Looking forward to know more.
Looking forward to know more.
AWhy don't you just stick to 8bit Adam if that is working for you? Why do you specifically want to use the D-Adaptation optimizers? They are slower. I can't help you with LoRA training unfortunately, I haven't trained any LoRA yet. Maybe that Russian guy was just lucky with his specific dataset he was training using the Facebook D-Adaptation optimizers because I have not been successful in getting them to work yet with regular Dreambooth training either. I guess we need to wait for the good Dr. to test them out and make a video for us.
T@Dr. Furkan Gözükara how to resume training using last gpt.pth in DLAS cmd interface?
Zah ok gotcha
Z@ashleyk quick question about class instances per image, is that based on the amount of images i'm using, or the amount of steps i'm taking? What would be a good number of class instances per image if i'm using 50 images?
Zok so i think i have everything set right now, using 50 images for my dataset this time
db_config.json3.98KB
Zexcept for the xformers, i'm gonna change that to default
ANo clue on this, Anzhc told you yesterday, but I don't know myself.
Zyeah so dataset yesterday was 15, and it used my prior preservation images, today i used a dataset of 50, with 50 class instances per image, and wanted to generate 1750 images, as soon as I dropped that to 10, it said my number of class instances per image is sufficient
Zso the dadapt with lora only uses like 5 gb of vram, wow
Zthat's impressive
AWhich D-Adapt optimizer specifically? In my testing Adan D-Adapt used 32GB of VRAM for regular Dreambooth training vs less than 24GB for any other optimizer.
ZNot mine. I'm using adan dadapt
ZAnd it uses 5gb for lora extended
Z4.7 with xformers
T@Dr. Furkan Gözükara how to resume training using last gpt.pth in DLAS cmd interface?
AWhy would you enable xformers for training if you don't need it?
AGuess the either:
1) The extensions implementation of the D-Adaptation Optimizers for full Dreambooth training is not correct.
2) There is a bug in the D-Adaptation Optimizers on Linux.
1) The extensions implementation of the D-Adaptation Optimizers for full Dreambooth training is not correct.
2) There is a bug in the D-Adaptation Optimizers on Linux.
AThe Dreambooth extension has too many bugs, has too much code committed to dev branch that isn't even tested at all, and is just generally bad quality and has the worst SDLC of any project in the entire world so I have totally given up on it.
Zwell i mean it KINDA works, so i only used 15 images, and it was somewhat resembling my wife, but not 100%, so I upped the sample images to 51, and retesting
Zplus it took 4 hours to do 1600 steps yestereday and didn't resemble my wife at all, i've actually gotten better results with xformers,
Zi'm doing 5100 total steps this time, 50 epochs, 51 images dataset, will do an x/y/z and compare and see how it looks
Zwhat's weird is even thouhg i set lora unet leaerning rate to 1 according to the article, it trains at 1.10e-6, but also his article says that adan dapation ignores l/r so i'm assumign thats' why
AThe D-Adaptation optimizers adapt the Learning Rate constantly, hence their name. If you install Tensorboard, you will be able to view the graph of the LR and you will see that its not constant.
AHere is a Tensorboard graph of the Adan D-Adaptation LR, the LR becomes constant eventually but it basically has a bit of a warm up at the beginning.
Zyeah i did about 5100 steps andd still doesn't even remotely resemble my subject, so either it's broken, or you have to do an insane amount of steps
ZI may increase the epochs to like 150 and see what happens
Zhow do i pull up that graph on mine?
Fcan you eloborate more what you mean
Fhaha i would also turn off but never looked for
Fjust solved that problem
Fgit checkout main
Fin dreambooth extension folder
Fhopefully coming soon
Fin advanced tab there is a file path which is .pth. it is original file. we give our checkpoint there
Fin past we had huge problem with xformers
Fit was training but not learning
Fxformers 0.0.16
Zis version 0.0.17 good?
Fit was
Fi dont know with latest scripts
Fdefinitely do a small test
Fand compare xformers vs not xformers
Zhow many num class instances per image should i use with a dataset of 50 images you think?
F50 is sweet spot i think
Fi am going to share 10k new class images on patreon today. for 2 new models
FrevAnimated_v122
FDreamShaper_6_NoVae
ASo it is possible to train a person into revAnimated? I tried before but failed.
Fi will try
Ffor styling