Software Engineering Courses (SECourses)•3y ago

Oh wow, looking forward to it Great videos by the way, have learnt so much in such a short amount of

Oh wow, looking forward to it
Great videos by the way, have learnt so much in such a short amount of time. You really are a credit to the community

Aashley Thanks, will try that, the old 1.0.14 release worked fine, but then they deleted...

Furkan Gözükara SECourses•6/7/23, 10:24 PM

lol

DDavid Ce Oh wow, looking forward to it Great videos by the way, have learnt so much in su...

Furkan Gözükara SECourses•6/7/23, 10:24 PM

ashley•6/7/23, 10:24 PM

New Facebook Optimizers are VERY slow

Furkan Gözükara SECourses•6/7/23, 10:25 PM

https://github.com/FurkanGozukara/Stable-Diffusion/blob/main/DreamBooth/DreamBooth-Working-Commits.md

GitHub

Stable-Diffusion/DreamBooth-Working-Commits.md at main · FurkanGozu...

Best Stable Diffusion and AI Tutorials, Guides, News, Tips and Tricks - Stable-Diffusion/DreamBooth-Working-Commits.md at main · FurkanGozukara/Stable-Diffusion

Aashley New Facebook Optimizers are VERY slow

Furkan Gözükara SECourses•6/7/23, 10:25 PM

havent tested yet

ashley•6/7/23, 10:25 PM

1.36 it/sec on 48GB GPU, its usually between 4-5 it/sec

Furkan Gözükara SECourses•6/7/23, 10:26 PM

wow really slow

Furkan Gözükara SECourses•6/7/23, 10:26 PM

did you increase batch size and gradient?

FFurkan Gözükara SECourses did you increase batch size and gradient?

ashley•6/7/23, 10:29 PM

No, was testing with Kohya_ss because Dreambooth extension is broken, but get bad results so not really bothering with these new optimizers for now.

Aashley No, was testing with Kohya_ss because Dreambooth extension is broken, but get ba...

Furkan Gözükara SECourses•6/7/23, 10:31 PM

you got bad results in kohya?

FFurkan Gözükara SECourses you got bad results in kohya?

ashley•6/7/23, 10:34 PM

with Adan D-Adapt, 8 bit Adam worked fine

ashley•6/7/23, 10:38 PM

testing Lion now and its almost 3 times faster

Isaac Brock•6/7/23, 10:41 PM

when im training in kohya i get "text_encoder\model.safetensors not found" does this mean my model isnt training using the text encoder learning?

Isaac Brock•6/7/23, 10:42 PM

I tried DAdaptation/DAdaptAdam and wasted a whole day. I get better results with AdamW 32 bit

Isaac Brock•6/7/23, 10:42 PM

constant 0.0001 Unet 0.00005 TE

Isaac Brock•6/7/23, 10:43 PM

and using noise offsets and other learning dampeners

Isaac Brock•6/7/23, 10:43 PM

the biggest jump in flexibility came when I lowered alpha to half dim or less

ashley•6/7/23, 10:47 PM

Yeah I also got bad results with it, but according to FB, LR should be 1

ashley•6/7/23, 10:47 PM

https://github.com/facebookresearch/dadaptation

GitHub

GitHub - facebookresearch/dadaptation: D-Adaptation for SGD, Adam a...

D-Adaptation for SGD, Adam and AdaGrad. Contribute to facebookresearch/dadaptation development by creating an account on GitHub.

Isaac Brock•6/7/23, 10:52 PM

yes i used 1 with it

ashley•6/7/23, 10:56 PM

Lion also works well, example dataset wearing a hat

ashley•6/7/23, 10:57 PM

cant get D-Adapt working properly

Townie•6/7/23, 11:36 PM

I'm getting an error when trying to train a model using the Colab link. Is there a way to remap the TensorRT accelerator?

TTownie I'm getting an error when trying to train a model using the Colab link. Is ther...

ashley•6/7/23, 11:40 PM

Its nothing to do with TensorRT, that is just a warning. The eror is because of a change in the diffusersdiffusers library.

Aashley Its nothing to do with TensorRT, that is just a warning. The eror is because of...

Townie•6/7/23, 11:41 PM

is this something that I can fix on my end, or does it need to be fixed by Dr. G?

TTownie is this something that I can fix on my end, or does it need to be fixed by Dr. G...

ashley•6/7/23, 11:41 PM

Not his code, not something he can fix

ashley•6/7/23, 11:41 PM

Or is it?

TTownie I'm getting an error when trying to train a model using the Colab link. Is ther...

ashley•6/7/23, 11:43 PM

Can you share the colab link with me so I can check it?

Townie•6/7/23, 11:44 PM

https://colab.research.google.com/github/FurkanGozukara/Stable-Diffusion/blob/main/DreamBooth/ShivamShriraoDreamBooth.ipynb#scrollTo=jjcSXTp-u-Eg

Google Colaboratory

Townie•6/7/23, 11:46 PM

I'm wondering if I need to remove one of the flags "use_8bit_adam" to make it work

TTownie I'm wondering if I need to remove one of the flags "use_8bit_adam" to make it wo...

ashley•6/7/23, 11:46 PM

No, that its bitsandbytes library not diffusers

ashley•6/7/23, 11:46 PM

Your error is from diffusers

TTownie I'm getting an error when trying to train a model using the Colab link. Is ther...

Furkan Gözükara SECourses•6/7/23, 11:47 PM

i will look for

ashley•6/7/23, 11:48 PM

Is that colab using huggingface diffusers or a fork of it?

ashley•6/7/23, 11:49 PM

You can actually remove logging_dirlogging_dir completely if you are not using Tensorboard

Furkan Gözükara SECourses•6/7/23, 11:50 PM

testing right now

Furkan Gözükara SECourses•6/7/23, 11:51 PM

must be accelerator version thing

Furkan Gözükara SECourses•6/7/23, 11:51 PM

i hate pythno

Furkan Gözükara SECourses•6/7/23, 11:51 PM

changing parameter names and functions

FFurkan Gözükara SECourses must be accelerator version thing

ashley•6/7/23, 11:53 PM

No its diffusers, how come the colab uses the ShivamShriraoShivamShrirao fork and not the original huggingfacehuggingface diffusers out of interest?

ashley•6/7/23, 11:54 PM

Not sure why it says the argument is unexpected, the code has it

ashley•6/7/23, 11:55 PM

Oh wait you are right, it is accelerate

ashley•6/7/23, 11:56 PM

These ML/AI guys just don't do releases properly, its not Python's fault

ashley•6/7/23, 11:56 PM

Python is an awesome language if people follow proper SDLC processes and don't randomly break things

ashley•6/7/23, 11:58 PM

ashley•6/7/23, 11:59 PM

Its still diffuser's fault for not locking in the version

Furkan Gözükara SECourses•6/8/23, 12:00 AM

2023-06-08 00:00:12.168156: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
/usr/local/lib/python3.10/dist-packages/accelerate/accelerator.py:258: FutureWarning: logging_dirlogging_dir is deprecated and will be removed in version 0.18.0 of

Accelerate. Use project_dirproject_dir instead.
warnings.warn(

Furkan Gözükara SECourses•6/8/23, 12:01 AM

18 or 17

Furkan Gözükara SECourses•6/8/23, 12:01 AM

will work testing

ashley•6/8/23, 12:05 AM

19 should also work, was just fully deprecated in 20

Oh wow, looking forward to it Great videos by the way, have learnt so much in such a short amount of

Similar Threads

Similar Threads

Similar Threads