Software Engineering Courses (SECourses)•3y ago

Im trying to find captioning techniques in your guides but couldnt get a clear idea. The SOTA you me

Im trying to find captioning techniques in your guides but couldnt get a clear idea. The SOTA you mentioned captions the image. Is that the one we need to use for Dreambooth/ Lora? Is Ohwx still best way to go to name someone? I saw some YT videos saying name to celebrity with high similarity is better way.

PPan_Pierdziadek btw should i use dreambooth tab or finetuning tab in kohya?

Furkan Gözükara SECourses•11/28/23, 3:12 PM

for dreambooth yes

Furkan Gözükara SECourses•11/28/23, 3:12 PM

dreambooth and fine tune same actually

Furkan Gözükara SECourses•11/28/23, 3:13 PM

only difference is using reg images

LLe Is there another option to D-ID that is not paid? D-ID is so expensive.

Furkan Gözükara SECourses•11/28/23, 3:13 PM

yes paid and expensive

Furkan Gözükara SECourses•11/28/23, 3:13 PM

i am searching for alternative open source

AAIGambino do you have a tutorial on regional prompting and lora extractions?

Furkan Gözükara SECourses•11/28/23, 3:13 PM

not yet

PPraveen Im trying to find captioning techniques in your guides but couldnt get a clear i...

Furkan Gözükara SECourses•11/28/23, 3:13 PM

if you are training a single person

Furkan Gözükara SECourses•11/28/23, 3:13 PM

you dont need caption

Furkan Gözükara SECourses•11/28/23, 3:13 PM

caption good for training hundreds of images

DR.Siva•11/28/23, 3:14 PM

if you train for face , the lora is pretty overwhelming at times and very underwhelming at times.. Here is what i found :

Test with lora weights , this will again change with each models , so running a xyz plot will help find a balanced weight
Adetailer! - again run a xyz plot for denoising strength. Last time when i created a lora , the strength i was comfortable with was 0.4 , now its 0.1-0.15 , very low. Even at 0.3 , its bringing very bad facial features , and beyong 8 , itsjust pasting face on the face like bad photoshop.
Balance with CFG value as well. This will give overall creativity vs very close to prompt creation
EAch model will give different results , so use same seed and try different models to see which is good. Also use different version of models to see which is good. Eg: collosus XL model gave me bad result in its latest version. so dont update to latest models , its not what we think , its not some software..

DR.Siva•11/28/23, 3:16 PM

Do remember sometime it be just a bad seed .. so dont come to any conclusion with just one output

DR.Siva•11/28/23, 3:32 PM

6.if you are confused with sampler, just use euler a and be contend with. it gives fast and good facial features

DR.Siva•11/28/23, 3:46 PM

7.Few things i felt which can change the outcome of face : Lora weight , Ohwx woman/man weight (use brackets to increase its weight or number) , Adetailer denoising strength , certain sampler , face restoration feature - i kept codeformers to 0.3 , face retoration back in adetailer. Will update if anything to be added

mikemenders•11/28/23, 4:16 PM

These are interesting things. In the last few Lora's I've coached women, I never adjusted the Lora's weight, it was always 1 and never modify CFG (always on 7). I had to use the Hires Fix to make the face even better, but many times it brought the person up nicely without it. But I'm tired of repeating the parameters, if you're interested, check out my comments here.

mikemenders•11/28/23, 4:25 PM

Yes, it's Lora, not Dreambooth, and I made it from plain Instagram pictures.

mikemenders•11/28/23, 4:30 PM

So a good Lora is not weak

mikemenders•11/28/23, 4:38 PM

But this is the end of my proof, I feel I have found the right Lora parameters, so good luck to all Lora tryers!

DR.Siva•11/28/23, 4:59 PM

pretty good, i havent activated teh hi-res for now

DR.Siva•11/28/23, 5:01 PM

this might sound very weird , but i feel automatic1111 starts to give better results after some image creation.. like it warms up after some images.. this is totally absurd thought.. just my feeling..

DR.Siva•11/28/23, 5:06 PM

Also if you are getting abnormal images , switch to 1024x1024 , it will start to give better results as its the base res. Also restarting auto1111 is also a not bad idea.

Mmikemenders These are interesting things. In the last few Lora's I've coached women, I never...

DR.Siva•11/28/23, 5:08 PM

what are your hires settinggs ?

aimademerich•11/28/23, 5:24 PM

Hello I ran into this issue during the kaggle install, ---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
Cell In[2], line 8
5 import threading
7 from flask import Flask
----> 8 from pyngrok import ngrok, conf
10 conf.get_default().auth_token = "---"
12 os.environ["FLASK_ENV"] = "development"

ModuleNotFoundError: No module named 'pyngrok'

[ ]:

!mkdir -p /kaggle/temp/models any suggestions?

aimademerich•11/28/23, 5:24 PM

i placed the token key in and this was the outcome

DDR.Siva what are your hires settinggs ?

mikemenders•11/28/23, 5:25 PM

My settings for Hires Fix: 15 steps with 0.4 noise and 4x_NMKD-Superscale-SP_178000_G, but the images above were taken without a hires fix or adetailer, I simply generated them.

Mmikemenders But this is the end of my proof, I feel I have found the right Lora parameters, ...

Benjamin•11/28/23, 7:09 PM

what are your next steps now?

Original message was deleted

AIGambino•11/28/23, 7:09 PM

what does your dataset look like for the face? I cannot get such a clear output with my dataset

BBenjamin what are your next steps now? 🙂

mikemenders•11/28/23, 7:20 PM

I have already trained in clothes and style. Prodigy has worked really well for me and I now know what to look out for on Tensorboard to get a good result. I'd like to make a fairly complex conceptual Lora. It's been on hold for a few weeks, but based on the experience of the last training sessions I might start again, but there's no urgency. It will also be a NSFW Lora, partly done by others, but not on a photo basis, but with animation. We'll see how it goes. It works as a separate Lora now, the challenge here will be the balance of kneading 4-5 concepts into one.

AAIGambino what does your dataset look like for the face? I cannot get such a clear output ...

mikemenders•11/28/23, 7:22 PM

To prepare a good image file, follow these steps:

Cut out the persons using Furkan's Yolo script.
Use CompreFace to select 25 images with a score of 0.99-1
Use Furkan's scaling/cropping script to create 512x512, 768x768 and 1024x1024 versions.
Train the model with Furkan's reg images, without caption.
With these you should get an absolutely good Lora.

mikemenders•11/28/23, 7:30 PM

And the lesson should not be lost:

you don't need many repetitions, 4-5 at most
not many pictures, 20-25, but I have trained style with 10.
the model is intelligent. From the reg pictures it knows it's a woman and from there it knows that women wear clothes or other things. So the model knows that the woman is what I'm training, so it takes her out of their environment. And because it knows the subject is a woman, it learns quickly.
When labelling, you have to train on a model that understands labels (otherwise the model will learn what you don't want them to learn)

mikemenders•11/28/23, 7:31 PM

Really unnecessary 10 or more repetitions. You're wasting your time if you're training Lora with lots of reps.

mikemenders•11/28/23, 7:34 PM

The reason I showed you the sample images is that it is possible to generate detailed images at 512x512 on SD 1.5 basis, which is faithful to the images you have loaded, without any help from Hires or Adetailer. Of course, the helpers are nice, but even the basic is good.

mikemenders•11/28/23, 7:35 PM

And with an SD 1.5 Lora, you can train with almost any smaller video card at home, with up to 6-8 GB of VRAM.

mikemenders•11/28/23, 7:37 PM

Yes, CompreFace is also important if you are training with images of strangers, because if you have images with a score of less than 0.99, your model will be bad. I very rarely let you train images with a score of 0.98 (if nothing else), because then the final results will also be scored 0.9 or below on close-up images.

mikemenders•11/28/23, 7:39 PM

And the script I posted last time is really good. I also trained with 64/64 weights on the Lora generating the images above. And I used it for styling as well. Just by setting d_coef I even trained SD 1.5 model with it to 768 px.

mikemenders•11/28/23, 7:41 PM

I look at the Tensorboard and I can see on the fly, even without sample images, that the d_coef value needs to be set stronger or weaker, and how strong the resulting Lora will be.

mikemenders•11/28/23, 7:42 PM

And Prodigy rewards good source images in return for results like this.

mikemenders•11/28/23, 7:45 PM

One more tip for CompreFace: choose a source image with both eyes visible and at least an upper body or closer face, and nothing covering the face (sunglasses, hands, whatever). And most important: whatever you want to generate. Since a person can have multiple faces (depending on age, environment), choose the image that is most typical of the person you would generate. Compare the others to that.

mikemenders•11/28/23, 7:48 PM

That's it, these are the tricks for a good result.

mikemenders•11/28/23, 7:58 PM

Fresh news for personal use only:
https://huggingface.co/stabilityai/sdxl-turbo

stabilityai/sdxl-turbo · Hugging Face

mikemenders•11/28/23, 7:59 PM

Test here:
https://clipdrop.co/stable-diffusion-turbo

Clipdrop - SDXL Turbo

mikemenders•11/28/23, 8:04 PM

"The generated images are of a fixed resolution (512x512 pix), and the model does not achieve perfect photorealism."

mikemenders•11/28/23, 8:05 PM

And on GitHub:
https://github.com/Stability-AI/generative-models

GitHub

GitHub - Stability-AI/generative-models: Generative Models by Stabi...

Generative Models by Stability AI. Contribute to Stability-AI/generative-models development by creating an account on GitHub.

schananas•11/28/23, 9:14 PM

Hi, does someone know if I can train dreambooth inpainting model in automatic1111 or kohya?

r1•11/28/23, 9:25 PM

your 24gb config disables xformers... isnt that super slow?

Mmikemenders To prepare a good image file, follow these steps: - Cut out the persons using Fu...

AIGambino•11/28/23, 9:26 PM

But will this know to center my person in the photo? It wont crop their head off?

AAIGambino But will this know to center my person in the photo? It wont crop their head off...

mikemenders•11/28/23, 9:32 PM

Furkan's script for Yolo does not cut the heads off, but cuts the people out of the picture first, so they are in focus. And the cropping script cuts out the images absolutely fine, so you don't have to worry about that, they help a lot in creating the perfect training set.

AIGambino•11/28/23, 9:32 PM

Does he discuss his scripts in a video by chance?

AAIGambino Does he discuss his scripts in a video by chance?

mikemenders•11/28/23, 9:38 PM

Here is his cropper script:
https://www.patreon.com/posts/sota-subject-and-88391247

mikemenders•11/28/23, 9:39 PM

This include yolo script too
And here is video:
https://www.youtube.com/watch?v=Fbuyu35TkE4

YouTubeSECourses

SOTA Image PreProcessing Scripts For Stable Diffusion Training - Au...

One of the most important aspect of Stable Diffusion training is the preparation of training images. In this tutorial video I will show you how to fully automatically preprocess training images with perfect zoom, crop and resize. These scripts will hugely improve your training success and accuracy.

Scripts Download Link

https://www.patreon...

Sschananas Hi, does someone know if I can train dreambooth inpainting model in automatic111...

Maelstromboni•11/28/23, 9:39 PM

If I understand you OK, it sounds like you want to switch the model used during Inpainting from the first model that generated the base image to a different, specialized model generated from Dreambooth.

If that's the case, you should be able to switch out models during Inpainting to use the other model OK.

Im trying to find captioning techniques in your guides but couldnt get a clear idea. The SOTA you me

Similar Threads

Similar Threads

Similar Threads