New Facebook Optimizers are VERY slow
New Facebook Optimizers are VERY slow
FGitHub
Best Stable Diffusion and AI Tutorials, Guides, News, Tips and Tricks - Stable-Diffusion/DreamBooth-Working-Commits.md at main · FurkanGozukara/Stable-Diffusion
Fhavent tested yet
A1.36 it/sec on 48GB GPU, its usually between 4-5 it/sec
Fwow really slow
Fdid you increase batch size and gradient?
ANo, was testing with Kohya_ss because Dreambooth extension is broken, but get bad results so not really bothering with these new optimizers for now.
Fyou got bad results in kohya?
Awith Adan D-Adapt, 8 bit Adam worked fine
Atesting Lion now and its almost 3 times faster
Iwhen im training in kohya i get "text_encoder\model.safetensors not found" does this mean my model isnt training using the text encoder learning?
II tried DAdaptation/DAdaptAdam and wasted a whole day. I get better results with AdamW 32 bit
Iconstant 0.0001 Unet 0.00005 TE
Iand using noise offsets and other learning dampeners
Ithe biggest jump in flexibility came when I lowered alpha to half dim or less
AYeah I also got bad results with it, but according to FB, LR should be 1
AGitHub
D-Adaptation for SGD, Adam and AdaGrad. Contribute to facebookresearch/dadaptation development by creating an account on GitHub.
Iyes i used 1 with it
ALion also works well, example dataset wearing a hat
Acant get D-Adapt working properly
TI'm getting an error when trying to train a model using the Colab link. Is there a way to remap the TensorRT accelerator?
AIts nothing to do with TensorRT, that is just a warning. The eror is because of a change in the
diffusersis this something that I can fix on my end, or does it need to be fixed by Dr. G?
ANot his code, not something he can fix
AOr is it?
ACan you share the colab link with me so I can check it?
TI'm wondering if I need to remove one of the flags "use_8bit_adam" to make it work
ANo, that its bitsandbytes library not diffusers
AYour error is from diffusers
Fi will look for
AIs that colab using huggingface diffusers or a fork of it?
AYou can actually remove
logging_dirtesting right now
Fmust be accelerator version thing
Fi hate pythno
Fchanging parameter names and functions
ANo its diffusers, how come the colab uses the
ShivamShriraohuggingfaceNot sure why it says the argument is unexpected, the code has it
AOh wait you are right, it is accelerate
AThese ML/AI guys just don't do releases properly, its not Python's fault
APython is an awesome language if people follow proper SDLC processes and don't randomly break things
A
AIts still diffuser's fault for not locking in the version
F2023-06-08 00:00:12.168156: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
/usr/local/lib/python3.10/dist-packages/accelerate/accelerator.py:258: FutureWarning:
Accelerate. Use
warnings.warn(
/usr/local/lib/python3.10/dist-packages/accelerate/accelerator.py:258: FutureWarning:
logging_dirproject_dirwarnings.warn(
F18 or 17
Fwill work testing
A19 should also work, was just fully deprecated in 20
Fok fixed
Fready
FGitHub
Best Stable Diffusion and AI Tutorials, Guides, News, Tips and Tricks - Stable-Diffusion/ShivamShriraoDreamBooth.ipynb at main · FurkanGozukara/Stable-Diffusion
