This is fantastic, I would like to do something like this when I finish my current project. Thanks f
This is fantastic, I would like to do something like this when I finish my current project. Thanks for sharing.
Fnice
Plet's see if I can document the process. There are several things that I learned along the way which greatly speed up the entire process.
All in all including the dataset and training 2 LoRAs this book took me ~15hrs.
From the current LoRA I could create new books in about 2 hrs. most of the time goes into generating and upscaling.
Though I'm hoping that with a dreambooth I might actually be able to create future stories on-demand via a small online platform.
Feels like we're really close to this (we've been planning and testing for ~6 months now, optimizing every aspect of the process)
All in all including the dataset and training 2 LoRAs this book took me ~15hrs.
From the current LoRA I could create new books in about 2 hrs. most of the time goes into generating and upscaling.
Though I'm hoping that with a dreambooth I might actually be able to create future stories on-demand via a small online platform.
Feels like we're really close to this (we've been planning and testing for ~6 months now, optimizing every aspect of the process)
PAt least Mike put on a jacket. Must be freezing with those shorts 
DWill you explain the character sheet
Pnot sure, what do you want to know?
DI understand the mike, willy, mike and will, what does contain character sheet
DI would like to try
PThe very first thing I do is create a character
When I have a prompt that does what I'm aiming for I extend it
I then run gens until I have several good starters. ( having a sheet generated is crucial for coherence)
You can then inpaint single poses and iterate until you have a decent turnaround and 2 - 3 faces.
side , diagonal, front , rear (all preferably from both sides)
smile, frown, scream, looking away...
All images then need to be fixed for perfection (photoshop and maybe inpainting or generate in Photoshop)
Now we create a new character sheet per hand and run that through img2img until we get some good coherence.
Now we upscale.
Upscale and highres fix the sheet,
slice the sheet and regenerate each pose, expression separately. (be careful to not loose coherence. loopbacks can help but I usually do each iteration by hand)
Finally, we have a dataset of 8 - 15 poses/expressions. All with exact same clothing, all on white background, (background removal is crucial at this point)
Good, we have a dataset: See screenshot with folder names. that's it, no captions, I just used the folder names on the first run, Keep it simple Stupid
I also added the character sheets to my dataset, to allow me to create better sheets with the first LoRA.
Then we move to version2. which is basically a rinse and repeat but with a lot better datasets, more poses, running, jumping, hand positions, but we still use white background on all images.
Get as much info in there as you can provide in best quality.
We still follow the rule: better less images than worse images
That's really the gist of it. No magic involved.
Jill the girl with short ginger hair and pale skin, a joyful character wearing a yellow sundress and black mary Janes in 1216x832When I have a prompt that does what I'm aiming for I extend it
Jill the girl with short ginger hair and pale skin, a joyful character wearing a yellow sundress and black mary Janes, consistent (character sheet, multiple views, various expressions, character turnaround:1.2) in 640x1536I then run gens until I have several good starters. ( having a sheet generated is crucial for coherence)
You can then inpaint single poses and iterate until you have a decent turnaround and 2 - 3 faces.
side , diagonal, front , rear (all preferably from both sides)
smile, frown, scream, looking away...
All images then need to be fixed for perfection (photoshop and maybe inpainting or generate in Photoshop)
Now we create a new character sheet per hand and run that through img2img until we get some good coherence.
Now we upscale.
Upscale and highres fix the sheet,
slice the sheet and regenerate each pose, expression separately. (be careful to not loose coherence. loopbacks can help but I usually do each iteration by hand)
Finally, we have a dataset of 8 - 15 poses/expressions. All with exact same clothing, all on white background, (background removal is crucial at this point)
Good, we have a dataset: See screenshot with folder names. that's it, no captions, I just used the folder names on the first run, Keep it simple Stupid
I also added the character sheets to my dataset, to allow me to create better sheets with the first LoRA.
Then we move to version2. which is basically a rinse and repeat but with a lot better datasets, more poses, running, jumping, hand positions, but we still use white background on all images.
Get as much info in there as you can provide in best quality.
We still follow the rule: better less images than worse images
That's really the gist of it. No magic involved.
SHi, I read on another videos that captioning helps a lot for the accurate prompt
DThank you very very much fir the explanation
DI will try first i have to learn then goes deep
Pthe first run is a waste LoRA. it is only use to get basic coherence. The second LoRa uses captioning.
...
That's why it's crucial to use white background here. Put the weights where you need them, don't waste the data. The model will be able to separate this
You can add contextual images and scenery when you train a final model on dreambooth.
Accept limitations, emphasize the strength.
So far from all my trainings this is the best result by far.
As mentioned before, This is strictly for illustrations, For photography or actually real people I use a similar but in various points different approach
Mike the boy, white backgroundMike the boy with hands in pocket, looking to side, white backgroundMike the boy with his dog Willy are running, white backgroundWilly the dog is barking, white background...
That's why it's crucial to use white background here. Put the weights where you need them, don't waste the data. The model will be able to separate this
You can add contextual images and scenery when you train a final model on dreambooth.
Accept limitations, emphasize the strength.
So far from all my trainings this is the best result by far.
As mentioned before, This is strictly for illustrations, For photography or actually real people I use a similar but in various points different approach
Pthe idea here is always the same: Describe what should be removable.
For Willy I never describe the blue collar and locket because it is part of Willy.
For Mike I never describe the clothing because they are part of Mike.
As I never want the character to have different clothing unless strictly prompted, this is crucial.
I will also add special images.
This will make outfit switching easier but keep his main outfit unless prompted otherwise
For Willy I never describe the blue collar and locket because it is part of Willy.
For Mike I never describe the clothing because they are part of Mike.
As I never want the character to have different clothing unless strictly prompted, this is crucial.
I will also add special images.
Mike the boy dressed as santa clausMike the boy as astronaut in a space suitThis will make outfit switching easier but keep his main outfit unless prompted otherwise
SI didn’t have any good results with LORA but with the regular dreambooth yes
PYeah, I'm really curious to see the results of Dreambooth on a bigger dataset.
�Guys
�i found free image generation ai model with no barriers
�(sometimes)
MA familiar trick. So first you create a "character generator" Lora, which is not flexible, then you use the generated images to create a tagged database and train a flexible Lora.
LThanks I was looking for that
LThis might be a dumb question, if you use dreambooth instead of Lora for the training, is the safetensors still used in the Lora folder like a Lora or the checkpoint folder? Or can you do either? And which is best if so?
G@Dr. Furkan Gözükara thanks for the quick response, Yes the file was acquired from patreon, my patreon emails is different than my discord email: aljohns
(venv) N:\AI\Stable_Diffusion\ClassImages\SECourses\Deepface>python
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug 1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>
(venv) N:\AI\Stable_Diffusion\ClassImages\SECourses\Deepface>python
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug 1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>
PYeah. Sort of. It’s all about efficiency. A good base should not take longer than a day including 2 trainings.
VUdemy also removed access to those that purchased and are offering a refund. Ridiculous.
PSome of you asked for it so I wrote a rather detailed article (warning dry content with no images, just plain detailed insights): https://blib.la/blog/mastering-character-consistency-in-stable-diffusion-sdxl-a-detailed-guide
Creating consistent characters in AI art, especially with tools like Stable Diffusion SDXL, requires a blend of creativity, precision, and the right techniques.
P
PTensor core models seem pointless for batch use...
PNot sure what all the hype is.
VMind adding RSS to your site? Would like to follow the blog.
MSpeaking of keywords, tonight I made a Lora model that I first trained with a keyword that was in the anime models but not (or had little chance of being) in the photo models. It performed relatively well on the photo models, but it confounded the anime models.
I then created a custom keyword from several words and it performed better on both model types.
And one more thing about repetition: I was training poses, and so I didn't use reg images. At first I doubled the number of repetitions, but I got a strong model. After going back to the "100/number of images" formula and using 4 reps instead of 8 (I have 25 images), I got a more flexible model. It was still a bit strong. I trained on Prodigy and 4/4 weights with 0.3 d_coef, I can't go below those values anymore.
I then created a custom keyword from several words and it performed better on both model types.
And one more thing about repetition: I was training poses, and so I didn't use reg images. At first I doubled the number of repetitions, but I got a strong model. After going back to the "100/number of images" formula and using 4 reps instead of 8 (I have 25 images), I got a more flexible model. It was still a bit strong. I trained on Prodigy and 4/4 weights with 0.3 d_coef, I can't go below those values anymore.
MOne more thing about flexibility: I can combine my own loras very well, so I can combine people with poses, styles, and they perform well on several models. A lot of the models that others have made have been so heavily trained that they simply can't be combined, and I'm thinking of generating images with them and re-creating them myself. Plus, some of them are so lousy that it takes tricks to get a good result. I admit I've never been a fan of weighting, because where you have to use a lot of weight, the model doesn't understand what I want. I know anime models love it, but I really find it unnecessary, but sometimes you have to use it for those crappy loras.
Ffull checkpoints goes into model folder and loras goes into lora folder
Fboth has .safetensors extension
Fhello. sorry for late reply. have you fixed your issue?
Fyep 100%. i am still waiting answer from them.
Fi tried only batch size 1 and getting over 70% speed up
G@Dr. Furkan Gözükara No I'm still having the error described above, any advice is greatly appreciated , thanks!
Fi would do this
Funinstall all python
Finstall as shown here
FYouTubeSECourses
Welcome to the Pillars of AI Application Libraries: How to Install Tutorial! 
Are you ready to embark on a journey of installing AI libraries and applications with ease? 
In this video, we'll guide you through the process of installing Python, Git, Visual Studio C++ Compile tools, and FFmpeg on your Windows 10 machine. 
We'll also show you ho...
Are you ready to embark on a journey of installing AI libraries and applications with ease? In this video, we'll guide you through the process of installing Python, Git, Visual Studio C++ Compile tools, and FFmpeg on your Windows 10 machine. We'll also show you ho...Falso install script directly into C drive
Flike c:/deepface
Fpretty sure it will get fixed
Gok I'll give that a try after my next checkpoint finishes
Gand report back, thanks
Fyour error
Findicating that
FFile "N:\AI\Stable_Diffusion\ClassImages\SECourses\Deepface\venv\lib\site-packages\tensorflow\python\tf2.py", line 21, in <module>
from tensorflow.python.platform import _pywrap_tf2
ImportError: DLL load failed while importing _pywrap_tf2: A dynamic link library (DLL) initialization routine failed.
from tensorflow.python.platform import _pywrap_tf2
ImportError: DLL load failed while importing _pywrap_tf2: A dynamic link library (DLL) initialization routine failed.