btw should i use dreambooth tab or finetuning tab in kohya?
btw should i use dreambooth tab or finetuning tab in kohya?
LThanks Professor 
LIs there another option to D-ID that is not paid? D-ID is so expensive.
Ado you have a tutorial on regional prompting and lora extractions?
PIm trying to find captioning techniques in your guides but couldnt get a clear idea. The SOTA you mentioned captions the image. Is that the one we need to use for Dreambooth/ Lora? Is Ohwx still best way to go to name someone? I saw some YT videos saying name to celebrity with high similarity is better way.
Ffor dreambooth yes
Fdreambooth and fine tune same actually
Fonly difference is using reg images
Fyes paid and expensive
Fi am searching for alternative open source
Fnot yet
Fif you are training a single person
Fyou dont need caption
Fcaption good for training hundreds of images
Dif you train for face , the lora is pretty overwhelming at times and very underwhelming at times.. Here is what i found :
- Test with lora weights , this will again change with each models , so running a xyz plot will help find a balanced weight
- Adetailer! - again run a xyz plot for denoising strength. Last time when i created a lora , the strength i was comfortable with was 0.4 , now its 0.1-0.15 , very low. Even at 0.3 , its bringing very bad facial features , and beyong 8 , itsjust pasting face on the face like bad photoshop.
- Balance with CFG value as well. This will give overall creativity vs very close to prompt creation
- EAch model will give different results , so use same seed and try different models to see which is good. Also use different version of models to see which is good. Eg: collosus XL model gave me bad result in its latest version. so dont update to latest models , its not what we think , its not some software..
D- Do remember sometime it be just a bad seed .. so dont come to any conclusion with just one output
D6.if you are confused with sampler, just use euler a and be contend with. it gives fast and good facial features
D7.Few things i felt which can change the outcome of face : Lora weight , Ohwx woman/man weight (use brackets to increase its weight or number) , Adetailer denoising strength , certain sampler , face restoration feature - i kept codeformers to 0.3 , face retoration back in adetailer. Will update if anything to be added
MThese are interesting things. In the last few Lora's I've coached women, I never adjusted the Lora's weight, it was always 1 and never modify CFG (always on 7). I had to use the Hires Fix to make the face even better, but many times it brought the person up nicely without it. But I'm tired of repeating the parameters, if you're interested, check out my comments here.
MYes, it's Lora, not Dreambooth, and I made it from plain Instagram pictures.
MSo a good Lora is not weak
MBut this is the end of my proof, I feel I have found the right Lora parameters, so good luck to all Lora tryers!
Dpretty good, i havent activated teh hi-res for now
Dthis might sound very weird , but i feel automatic1111 starts to give better results after some image creation.. like it warms up after some images.. this is totally absurd thought.. just my feeling..
DAlso if you are getting abnormal images , switch to 1024x1024 , it will start to give better results as its the base res. Also restarting auto1111 is also a not bad idea.
Dwhat are your hires settinggs ?
AHello I ran into this issue during the kaggle install, ---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
Cell In[2], line 8
5 import threading
7 from flask import Flask
----> 8 from pyngrok import ngrok, conf
10 conf.get_default().auth_token = "---"
12 os.environ["FLASK_ENV"] = "development"
ModuleNotFoundError: No module named 'pyngrok'
[ ]:
!mkdir -p /kaggle/temp/models any suggestions?
ModuleNotFoundError Traceback (most recent call last)
Cell In[2], line 8
5 import threading
7 from flask import Flask
----> 8 from pyngrok import ngrok, conf
10 conf.get_default().auth_token = "---"
12 os.environ["FLASK_ENV"] = "development"
ModuleNotFoundError: No module named 'pyngrok'
[ ]:
!mkdir -p /kaggle/temp/models any suggestions?
Ai placed the token key in and this was the outcome
MMy settings for Hires Fix: 15 steps with 0.4 noise and 4x_NMKD-Superscale-SP_178000_G, but the images above were taken without a hires fix or adetailer, I simply generated them.
Bwhat are your next steps now?
Awhat does your dataset look like for the face? I cannot get such a clear output with my dataset
MI have already trained in clothes and style. Prodigy has worked really well for me and I now know what to look out for on Tensorboard to get a good result. I'd like to make a fairly complex conceptual Lora. It's been on hold for a few weeks, but based on the experience of the last training sessions I might start again, but there's no urgency. It will also be a NSFW Lora, partly done by others, but not on a photo basis, but with animation. We'll see how it goes. It works as a separate Lora now, the challenge here will be the balance of kneading 4-5 concepts into one.
MTo prepare a good image file, follow these steps:
- Cut out the persons using Furkan's Yolo script.
- Use CompreFace to select 25 images with a score of 0.99-1
- Use Furkan's scaling/cropping script to create 512x512, 768x768 and 1024x1024 versions.
- Train the model with Furkan's reg images, without caption.
With these you should get an absolutely good Lora.
MAnd the lesson should not be lost:
- you don't need many repetitions, 4-5 at most
- not many pictures, 20-25, but I have trained style with 10.
- the model is intelligent. From the reg pictures it knows it's a woman and from there it knows that women wear clothes or other things. So the model knows that the woman is what I'm training, so it takes her out of their environment. And because it knows the subject is a woman, it learns quickly.
- When labelling, you have to train on a model that understands labels (otherwise the model will learn what you don't want them to learn)
MReally unnecessary 10 or more repetitions. You're wasting your time if you're training Lora with lots of reps.
MThe reason I showed you the sample images is that it is possible to generate detailed images at 512x512 on SD 1.5 basis, which is faithful to the images you have loaded, without any help from Hires or Adetailer. Of course, the helpers are nice, but even the basic is good.
MAnd with an SD 1.5 Lora, you can train with almost any smaller video card at home, with up to 6-8 GB of VRAM.
MYes, CompreFace is also important if you are training with images of strangers, because if you have images with a score of less than 0.99, your model will be bad. I very rarely let you train images with a score of 0.98 (if nothing else), because then the final results will also be scored 0.9 or below on close-up images.
MAnd the script I posted last time is really good. I also trained with 64/64 weights on the Lora generating the images above. And I used it for styling as well. Just by setting d_coef I even trained SD 1.5 model with it to 768 px.
MI look at the Tensorboard and I can see on the fly, even without sample images, that the d_coef value needs to be set stronger or weaker, and how strong the resulting Lora will be.
MAnd Prodigy rewards good source images in return for results like this.
MOne more tip for CompreFace: choose a source image with both eyes visible and at least an upper body or closer face, and nothing covering the face (sunglasses, hands, whatever). And most important: whatever you want to generate. Since a person can have multiple faces (depending on age, environment), choose the image that is most typical of the person you would generate. Compare the others to that.
MThat's it, these are the tricks for a good result.
M
MClipdrop - SDXL Turbo
M"The generated images are of a fixed resolution (512x512 pix), and the model does not achieve perfect photorealism."
MGitHub
Generative Models by Stability AI. Contribute to Stability-AI/generative-models development by creating an account on GitHub.
SHi, does someone know if I can train dreambooth inpainting model in automatic1111 or kohya?
Ryour 24gb config disables xformers... isnt that super slow?
ABut will this know to center my person in the photo? It wont crop their head off?
MFurkan's script for Yolo does not cut the heads off, but cuts the people out of the picture first, so they are in focus. And the cropping script cuts out the images absolutely fine, so you don't have to worry about that, they help a lot in creating the perfect training set.