Software Engineering Courses (SECourses)•3y ago

I am using sd-scripts directly for training and then using diffusers for inference. I’ve narrowed d

I am using sd-scripts directly for training and then using diffusers for inference.

I’ve narrowed down the issue to bf16 mixed precision setting not loading the vae as half precision.

stillmotionOP•10/11/23, 7:40 PM

Not clear to me why the vae loading doesn’t support bf16 and instead loads the full float version

Sstillmotion I am using sd-scripts directly for training and then using diffusers for inferen...

Furkan Gözükara SECourses•10/11/23, 7:41 PM

diffusers inference problematic

Furkan Gözükara SECourses•10/11/23, 7:41 PM

i am giving consultation to 1 company

Furkan Gözükara SECourses•10/11/23, 7:41 PM

they failed to replicate automatic1111 yet

stillmotionOP•10/11/23, 7:41 PM

Possibly, but seems like the issue is with the training at the moment

stillmotionOP•10/11/23, 7:42 PM

Was going to change the mixed precision to use fp16, but it’s incompatible with full_bf16 training.

stillmotionOP•10/11/23, 7:43 PM

So I’m either missing something or the vae for bf16 training requires full precision

Furkan Gözükara SECourses•10/11/23, 7:43 PM

vae is fp16 or fp32

Furkan Gözükara SECourses•10/11/23, 7:43 PM

bf16 vae probably wont work

Furkan Gözükara SECourses•10/11/23, 7:43 PM

not engough precision

Furkan Gözükara SECourses•10/11/23, 7:43 PM

also do you have bf16 vae?

stillmotionOP•10/11/23, 7:44 PM

I don’t. I’m just using the one from hugging face that you linked to in a tutorial.

Sstillmotion I don’t. I’m just using the one from hugging face that you linked to in a tutori...

Furkan Gözükara SECourses•10/11/23, 7:45 PM

that is fp16

Furkan Gözükara SECourses•10/11/23, 7:45 PM

not bf16

stillmotionOP•10/11/23, 7:46 PM

I’ll have to look at your dreambooth settings again (on my phone), are you using full bf16 training?

stillmotionOP•10/11/23, 7:47 PM

And mixed precision bf16?

stillmotionOP•10/11/23, 7:47 PM

Or just saving the model as bf16?

Sstillmotion Or just saving the model as bf16?

Furkan Gözükara SECourses•10/11/23, 7:54 PM

dreambooth training is full bf16

Furkan Gözükara SECourses•10/11/23, 7:54 PM

but that is indepented of vae when we do inference

Furkan Gözükara SECourses•10/11/23, 7:54 PM

you can also use embedded vae

Furkan Gözükara SECourses•10/11/23, 7:54 PM

we dont train vae

stillmotionOP•10/11/23, 8:01 PM

ah ok, so you don't set a different vae when training? You just use the one embedded in the model?

Sstillmotion ah ok, so you don't set a different vae when training? You just use the one embe...

Furkan Gözükara SECourses•10/11/23, 8:01 PM

yep

stillmotionOP•10/11/23, 8:02 PM

that is probably where my confusion is coming from -- i'm setting the one that you specified for inference for training also

Sstillmotion that is probably where my confusion is coming from -- i'm setting the one that y...

Furkan Gözükara SECourses•10/11/23, 8:02 PM

no i never set vae when training

thesilvermoth•10/11/23, 8:40 PM

Hello! Which extension do you recommend for face swapping? Only image, no video

Tthesilvermoth Hello! Which extension do you recommend for face swapping? Only image, no video

Furkan Gözükara SECourses•10/11/23, 8:41 PM

i only used roop but not as an extension

Furkan Gözükara SECourses•10/11/23, 9:04 PM

https://twitter.com/GozukaraFurkan/status/1712212069796188398

Furkan Gözükara SECourses•10/11/23, 9:23 PM

https://twitter.com/GozukaraFurkan/status/1712217282905014723

Stefanoid9343•10/11/23, 9:37 PM

Hi @Dr. Furkan Gözükara I have the following message on kaggle and I cannot type "y" N: This must be accepted explicitly before updates for this repository can be applied. See apt-secure(8) manpage for details.
Do you want to accept these changes and continue updating from this repository? [y/N]

land23•10/11/23, 10:15 PM

@Dr. Furkan Gözükara so, what's the BEST dreambooth model you've ever trained?

land23•10/11/23, 10:24 PM

I'm looking at this post: https://www.patreon.com/posts/best-stable-1-5-90307694 - but did you only train it for 4 epochs and 1,000 steps?

Patreon

Best Stable Diffusion 1.5 Based Models Training Config - Parameters...

Get more from SECourses: Tutorials, Guides, Resources, Training, MidJourney, Voice Clone, TTS, ChatGPT, GPT, LLM, Scripts by Furkan Gözükara on Patreon