Good night group any of you knows how to create those talking avatars with videos? I would like to c
Good night group any of you knows how to create those talking avatars with videos? I would like to create those but with training images
G@Dr. Furkan Gözükara trying to apply this video: https://www.youtube.com/watch?v=343I11mhnXs however when running the script I'm getting the following error
File "N:\AI\Stable_Diffusion\ClassImages\SECourses\Deepface\venv\lib\site-packages\tensorflow\python\tf2.py", line 21, in <module>
from tensorflow.python.platform import _pywrap_tf2
ImportError: DLL load failed while importing _pywrap_tf2: A dynamic link library (DLL) initialization routine failed.
I've already tried installing the zlibwapi.dll into the scripts folder. I have an older computer with an NVIDA 3060 any assitance would be greatly appreciated and thanks for all your very detailed explanations!
File "N:\AI\Stable_Diffusion\ClassImages\SECourses\Deepface\venv\lib\site-packages\tensorflow\python\tf2.py", line 21, in <module>
from tensorflow.python.platform import _pywrap_tf2
ImportError: DLL load failed while importing _pywrap_tf2: A dynamic link library (DLL) initialization routine failed.
I've already tried installing the zlibwapi.dll into the scripts folder. I have an older computer with an NVIDA 3060 any assitance would be greatly appreciated and thanks for all your very detailed explanations!
YouTubeSECourses
If you are also getting tired of trying to find good images among thousands of generated images you don't have to anymore. By using #DeepFace AI library, you can sort images by their similarity to your target images and quickly find the best Stable Diffusion #DreamBooth LoRA trained model generated images. I am explaining everything step by step...
LThe old comfy install I had isn't booting up, I tried a fresh install and had the same issue. It's probably a dumb item. I hadn't started it since we changed over Python.
HNice work! How are you going about maintaining the consistency in the character?
PIt’s already trained like this. I can now prompt Mike the boy with or without his dog Willy or just Willy the dog alone. The character stays consistent most of the time. Good enough to pick from a batch of 4 ATM. I will train at least one more version, but I’ll try dreambooth next and add a lot more images which I can now easily generate. The first 24 pages book (prototype) is already done.
HOk, I was doing something similar with Dreambooth but havent been able to do so. If you are following any tutorial or decide to make one or have any tips :D, pls do share.
I want to create a comic strip with few characters which can remain consistent across their poses, expressions etc. So looking for ideas around it!
I want to create a comic strip with few characters which can remain consistent across their poses, expressions etc. So looking for ideas around it!
PYeah. I might create a tutorial about the process at some point. Not sure if video or blog or even both. . Also my training data and Lora will be open-sourced first research (no commercial use)
PFirst prototype.
PMy process allows creating new character sheets (I trained character sheets, solo images and coupled images.
Looks like it was extremely effective
Looks like it was extremely effective
HYeah looks great. So you trained this model on Dreambooth? Or is it Lora?
Pjust a standard LoRA with my usual settings. It is v2.
All training images are 4k - 8k, and captioned in a very similar manner
.. and so on.
See the attached screenshot on repeats. (the caption is done via text files, so only the number is used from those folder names)
I trained 10 epochs of which 7 - 10 at 0.7 - 1.0 weight are actually usable.
- version was trained on 14 images (including solo, coupled and character sheets) all on white background
- version included 35 images with more poses (including solo, coupled and character sheets) all on white background
All training images are 4k - 8k, and captioned in a very similar manner
Mike the boyWilly the dogMike the boy with his dog WillyMike the boy is runningMike the boy with hands in pocket, upper bodyMike the boy with mouth open, screaming, close-up.. and so on.
See the attached screenshot on repeats. (the caption is done via text files, so only the number is used from those folder names)
I trained 10 epochs of which 7 - 10 at 0.7 - 1.0 weight are actually usable.
Hgot it, thanks for sharing 
I am using img2img with Cheyenne model using Canny controlnet to convert my imgs to cartoon. It is looking good so far. But next step is to train models for more flexibility.
I am using img2img with Cheyenne model using Canny controlnet to convert my imgs to cartoon. It is looking good so far. But next step is to train models for more flexibility.
PI only trained on whimsical watercolor images (as seen in the results). When I use a photography style on them I get these (1-shot) which is not perfect, but still quite impressive, considering I never trained any photos
Fi worked on a repo but its resolution too low
FGitHub
[CVPR 2022] Thin-Plate Spline Motion Model for Image Animation. - GitHub - yoyo-nb/Thin-Plate-Spline-Motion-Model: [CVPR 2022] Thin-Plate Spline Motion Model for Image Animation.
Fhello welcome. you downloaded from our patreon right?
Fcan you tell me what do you see when you open cmd and type python?
Fyou can activate venv and install scipy manually
FFso nice
Ptesting some additional styles on Mike and Willy
P
XThis is fantastic, I would like to do something like this when I finish my current project. Thanks for sharing.
Fnice
Plet's see if I can document the process. There are several things that I learned along the way which greatly speed up the entire process.
All in all including the dataset and training 2 LoRAs this book took me ~15hrs.
From the current LoRA I could create new books in about 2 hrs. most of the time goes into generating and upscaling.
Though I'm hoping that with a dreambooth I might actually be able to create future stories on-demand via a small online platform.
Feels like we're really close to this (we've been planning and testing for ~6 months now, optimizing every aspect of the process)
All in all including the dataset and training 2 LoRAs this book took me ~15hrs.
From the current LoRA I could create new books in about 2 hrs. most of the time goes into generating and upscaling.
Though I'm hoping that with a dreambooth I might actually be able to create future stories on-demand via a small online platform.
Feels like we're really close to this (we've been planning and testing for ~6 months now, optimizing every aspect of the process)
PAt least Mike put on a jacket. Must be freezing with those shorts 
DWill you explain the character sheet
Pnot sure, what do you want to know?
DI understand the mike, willy, mike and will, what does contain character sheet
DI would like to try
PThe very first thing I do is create a character
When I have a prompt that does what I'm aiming for I extend it
I then run gens until I have several good starters. ( having a sheet generated is crucial for coherence)
You can then inpaint single poses and iterate until you have a decent turnaround and 2 - 3 faces.
side , diagonal, front , rear (all preferably from both sides)
smile, frown, scream, looking away...
All images then need to be fixed for perfection (photoshop and maybe inpainting or generate in Photoshop)
Now we create a new character sheet per hand and run that through img2img until we get some good coherence.
Now we upscale.
Upscale and highres fix the sheet,
slice the sheet and regenerate each pose, expression separately. (be careful to not loose coherence. loopbacks can help but I usually do each iteration by hand)
Finally, we have a dataset of 8 - 15 poses/expressions. All with exact same clothing, all on white background, (background removal is crucial at this point)
Good, we have a dataset: See screenshot with folder names. that's it, no captions, I just used the folder names on the first run, Keep it simple Stupid
I also added the character sheets to my dataset, to allow me to create better sheets with the first LoRA.
Then we move to version2. which is basically a rinse and repeat but with a lot better datasets, more poses, running, jumping, hand positions, but we still use white background on all images.
Get as much info in there as you can provide in best quality.
We still follow the rule: better less images than worse images
That's really the gist of it. No magic involved.
Jill the girl with short ginger hair and pale skin, a joyful character wearing a yellow sundress and black mary Janes in 1216x832When I have a prompt that does what I'm aiming for I extend it
Jill the girl with short ginger hair and pale skin, a joyful character wearing a yellow sundress and black mary Janes, consistent (character sheet, multiple views, various expressions, character turnaround:1.2) in 640x1536I then run gens until I have several good starters. ( having a sheet generated is crucial for coherence)
You can then inpaint single poses and iterate until you have a decent turnaround and 2 - 3 faces.
side , diagonal, front , rear (all preferably from both sides)
smile, frown, scream, looking away...
All images then need to be fixed for perfection (photoshop and maybe inpainting or generate in Photoshop)
Now we create a new character sheet per hand and run that through img2img until we get some good coherence.
Now we upscale.
Upscale and highres fix the sheet,
slice the sheet and regenerate each pose, expression separately. (be careful to not loose coherence. loopbacks can help but I usually do each iteration by hand)
Finally, we have a dataset of 8 - 15 poses/expressions. All with exact same clothing, all on white background, (background removal is crucial at this point)
Good, we have a dataset: See screenshot with folder names. that's it, no captions, I just used the folder names on the first run, Keep it simple Stupid
I also added the character sheets to my dataset, to allow me to create better sheets with the first LoRA.
Then we move to version2. which is basically a rinse and repeat but with a lot better datasets, more poses, running, jumping, hand positions, but we still use white background on all images.
Get as much info in there as you can provide in best quality.
We still follow the rule: better less images than worse images
That's really the gist of it. No magic involved.
SHi, I read on another videos that captioning helps a lot for the accurate prompt
DThank you very very much fir the explanation
DI will try first i have to learn then goes deep
Pthe first run is a waste LoRA. it is only use to get basic coherence. The second LoRa uses captioning.
...
That's why it's crucial to use white background here. Put the weights where you need them, don't waste the data. The model will be able to separate this
You can add contextual images and scenery when you train a final model on dreambooth.
Accept limitations, emphasize the strength.
So far from all my trainings this is the best result by far.
As mentioned before, This is strictly for illustrations, For photography or actually real people I use a similar but in various points different approach
Mike the boy, white backgroundMike the boy with hands in pocket, looking to side, white backgroundMike the boy with his dog Willy are running, white backgroundWilly the dog is barking, white background...
That's why it's crucial to use white background here. Put the weights where you need them, don't waste the data. The model will be able to separate this
You can add contextual images and scenery when you train a final model on dreambooth.
Accept limitations, emphasize the strength.
So far from all my trainings this is the best result by far.
As mentioned before, This is strictly for illustrations, For photography or actually real people I use a similar but in various points different approach
Pthe idea here is always the same: Describe what should be removable.
For Willy I never describe the blue collar and locket because it is part of Willy.
For Mike I never describe the clothing because they are part of Mike.
As I never want the character to have different clothing unless strictly prompted, this is crucial.
I will also add special images.
This will make outfit switching easier but keep his main outfit unless prompted otherwise
For Willy I never describe the blue collar and locket because it is part of Willy.
For Mike I never describe the clothing because they are part of Mike.
As I never want the character to have different clothing unless strictly prompted, this is crucial.
I will also add special images.
Mike the boy dressed as santa clausMike the boy as astronaut in a space suitThis will make outfit switching easier but keep his main outfit unless prompted otherwise
SI didn’t have any good results with LORA but with the regular dreambooth yes
PYeah, I'm really curious to see the results of Dreambooth on a bigger dataset.
�Guys
�i found free image generation ai model with no barriers
�(sometimes)
MA familiar trick. So first you create a "character generator" Lora, which is not flexible, then you use the generated images to create a tagged database and train a flexible Lora.
LThanks I was looking for that
LThis might be a dumb question, if you use dreambooth instead of Lora for the training, is the safetensors still used in the Lora folder like a Lora or the checkpoint folder? Or can you do either? And which is best if so?
G@Dr. Furkan Gözükara thanks for the quick response, Yes the file was acquired from patreon, my patreon emails is different than my discord email: aljohns
(venv) N:\AI\Stable_Diffusion\ClassImages\SECourses\Deepface>python
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug 1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>
(venv) N:\AI\Stable_Diffusion\ClassImages\SECourses\Deepface>python
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug 1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>
PYeah. Sort of. It’s all about efficiency. A good base should not take longer than a day including 2 trainings.
VUdemy also removed access to those that purchased and are offering a refund. Ridiculous.
PSome of you asked for it so I wrote a rather detailed article (warning dry content with no images, just plain detailed insights): https://blib.la/blog/mastering-character-consistency-in-stable-diffusion-sdxl-a-detailed-guide
Creating consistent characters in AI art, especially with tools like Stable Diffusion SDXL, requires a blend of creativity, precision, and the right techniques.
P
PTensor core models seem pointless for batch use...
PNot sure what all the hype is.