Software Engineering Courses (SECourses)•3y ago

just a standard LoRA with my usual settings. It is v2. 1. version was trained on 14 images (includi

just a standard LoRA with my usual settings. It is v2.

version was trained on 14 images (including solo, coupled and character sheets) all on white background
version included 35 images with more poses (including solo, coupled and character sheets) all on white background

Next will be dreambooth, with ~ 60 - 100 images. Will have to test what works best.

All training images are 4k - 8k, and captioned in a very similar manner

Mike the boy

Mike the boy

Willy the dog

Willy the dog

Mike the boy with his dog Willy

Mike the boy with his dog Willy

Mike the boy is running

Mike the boy is running

Mike the boy with hands in pocket, upper body

Mike the boy with hands in pocket, upper body

Mike the boy with mouth open, screaming, close-up

Mike the boy with mouth open, screaming, close-up

.. and so on.

See the attached screenshot on repeats. (the caption is done via text files, so only the number is used from those folder names)

I trained 10 epochs of which 7 - 10 at 0.7 - 1.0 weight are actually usable.

Hustle with AI•12/18/23, 10:16 AM

got it, thanks for sharing

I am using img2img with Cheyenne model using Canny controlnet to convert my imgs to cartoon. It is looking good so far. But next step is to train models for more flexibility.

HHustle with AI got it, thanks for sharing 👍 I am using img2img with Cheyenne model using C...

Pflanzen die TanzenOP•12/18/23, 10:44 AM

I only trained on whimsical watercolor images (as seen in the results). When I use a photography style on them I get these (1-shot) which is not perfect, but still quite impressive, considering I never trained any photos

SStefanoid9343 Good night group any of you knows how to create those talking avatars with video...

Furkan Gözükara SECourses•12/18/23, 10:50 AM

i worked on a repo but its resolution too low

Furkan Gözükara SECourses•12/18/23, 10:50 AM

https://github.com/yoyo-nb/Thin-Plate-Spline-Motion-Model

GitHub

GitHub - yoyo-nb/Thin-Plate-Spline-Motion-Model: [CVPR 2022] Thin-P...

[CVPR 2022] Thin-Plate Spline Motion Model for Image Animation. - GitHub - yoyo-nb/Thin-Plate-Spline-Motion-Model: [CVPR 2022] Thin-Plate Spline Motion Model for Image Animation.

Ggoalpwr @Dr. Furkan Gözükara trying to apply this video: https://www.youtube.com/watch?...

Furkan Gözükara SECourses•12/18/23, 10:51 AM

hello welcome. you downloaded from our patreon right?

Furkan Gözükara SECourses•12/18/23, 10:51 AM

can you tell me what do you see when you open cmd and type python?

LLorelai AI The old comfy install I had isn't booting up, I tried a fresh install and had th...

Furkan Gözükara SECourses•12/18/23, 10:51 AM

you can activate venv and install scipy manually

Furkan Gözükara SECourses•12/18/23, 10:52 AM

i also have auto installer : https://www.patreon.com/posts/92013455

PPflanzen die Tanzen First prototype.

Furkan Gözükara SECourses•12/18/23, 10:52 AM

so nice

Pflanzen die TanzenOP•12/18/23, 10:53 AM

testing some additional styles on Mike and Willy

Pflanzen die TanzenOP•12/18/23, 11:36 AM

PPflanzen die Tanzen First prototype.

Xylber•12/18/23, 12:08 PM

This is fantastic, I would like to do something like this when I finish my current project. Thanks for sharing.

Furkan Gözükara SECourses•12/18/23, 12:27 PM

nice

XXylber This is fantastic, I would like to do something like this when I finish my curre...

Pflanzen die TanzenOP•12/18/23, 12:41 PM

let's see if I can document the process. There are several things that I learned along the way which greatly speed up the entire process.
All in all including the dataset and training 2 LoRAs this book took me ~15hrs.
From the current LoRA I could create new books in about 2 hrs. most of the time goes into generating and upscaling.

Though I'm hoping that with a dreambooth I might actually be able to create future stories on-demand via a small online platform.

Feels like we're really close to this (we've been planning and testing for ~6 months now, optimizing every aspect of the process)

Pflanzen die TanzenOP•12/18/23, 12:59 PM

At least Mike put on a jacket. Must be freezing with those shorts

PPflanzen die Tanzen just a standard LoRA with my usual settings. It is v2. 1. version was trained o...

Deleted User•12/18/23, 1:12 PM

Will you explain the character sheet

DDeleted User Will you explain the character sheet

Pflanzen die TanzenOP•12/18/23, 1:15 PM

not sure, what do you want to know?

PPflanzen die Tanzen not sure, what do you want to know?

Deleted User•12/18/23, 1:17 PM

I understand the mike, willy, mike and will, what does contain character sheet

Deleted User•12/18/23, 1:17 PM

I would like to try

Pflanzen die TanzenOP•12/18/23, 1:31 PM

The very first thing I do is create a character

Jill the girl with short ginger hair and pale skin, a joyful character wearing a yellow sundress and black mary Janes

Jill the girl with short ginger hair and pale skin, a joyful character wearing a yellow sundress and black mary Janes

in 1216x832

When I have a prompt that does what I'm aiming for I extend it

Jill the girl with short ginger hair and pale skin, a joyful character wearing a yellow sundress and black mary Janes, consistent (character sheet, multiple views, various expressions, character turnaround:1.2)

Jill the girl with short ginger hair and pale skin, a joyful character wearing a yellow sundress and black mary Janes, consistent (character sheet, multiple views, various expressions, character turnaround:1.2)

in 640x1536

I then run gens until I have several good starters. ( having a sheet generated is crucial for coherence)

You can then inpaint single poses and iterate until you have a decent turnaround and 2 - 3 faces.

side , diagonal, front , rear (all preferably from both sides)
smile, frown, scream, looking away...

All images then need to be fixed for perfection (photoshop and maybe inpainting or generate in Photoshop)
Now we create a new character sheet per hand and run that through img2img until we get some good coherence.
Now we upscale.
Upscale and highres fix the sheet,
slice the sheet and regenerate each pose, expression separately. (be careful to not loose coherence. loopbacks can help but I usually do each iteration by hand)

Finally, we have a dataset of 8 - 15 poses/expressions. All with exact same clothing, all on white background, (background removal is crucial at this point)

Good, we have a dataset: See screenshot with folder names. that's it, no captions, I just used the folder names on the first run, Keep it simple Stupid

I also added the character sheets to my dataset, to allow me to create better sheets with the first LoRA.

Then we move to version2. which is basically a rinse and repeat but with a lot better datasets, more poses, running, jumping, hand positions, but we still use white background on all images.
Get as much info in there as you can provide in best quality.
We still follow the rule: better less images than worse images

That's really the gist of it. No magic involved.

PPflanzen die Tanzen The very first thing I do is create a character `Jill the girl with short ginge...

Stefanoid9343•12/18/23, 1:34 PM

Hi, I read on another videos that captioning helps a lot for the accurate prompt

PPflanzen die Tanzen The very first thing I do is create a character `Jill the girl with short ginge...

Deleted User•12/18/23, 1:34 PM

Thank you very very much fir the explanation

Deleted User•12/18/23, 1:36 PM

I will try first i have to learn then goes deep

SStefanoid9343 Hi, I read on another videos that captioning helps a lot for the accurate prompt

Pflanzen die TanzenOP•12/18/23, 1:39 PM

the first run is a waste LoRA. it is only use to get basic coherence. The second LoRa uses captioning.

Mike the boy, white background

Mike the boy, white background

Mike the boy with hands in pocket, looking to side, white background

Mike the boy with hands in pocket, looking to side, white background

Mike the boy with his dog Willy are running, white background

Mike the boy with his dog Willy are running, white background

Willy the dog is barking, white background

Willy the dog is barking, white background

...

That's why it's crucial to use white background here. Put the weights where you need them, don't waste the data. The model will be able to separate this

You can add contextual images and scenery when you train a final model on dreambooth.

Accept limitations, emphasize the strength.

So far from all my trainings this is the best result by far.

As mentioned before, This is strictly for illustrations, For photography or actually real people I use a similar but in various points different approach

Pflanzen die TanzenOP•12/18/23, 1:43 PM

the idea here is always the same: Describe what should be removable.
For Willy I never describe the blue collar and locket because it is part of Willy.
For Mike I never describe the clothing because they are part of Mike.

As I never want the character to have different clothing unless strictly prompted, this is crucial.

I will also add special images.

Mike the boy dressed as santa claus

Mike the boy dressed as santa claus

Mike the boy as astronaut in a space suit

Mike the boy as astronaut in a space suit

This will make outfit switching easier but keep his main outfit unless prompted otherwise

PPflanzen die Tanzen the idea here is always the same: Describe what should be removable. For Willy ...

Stefanoid9343•12/18/23, 2:01 PM

I didn’t have any good results with LORA but with the regular dreambooth yes

SStefanoid9343 I didn’t have any good results with LORA but with the regular dreambooth yes

Pflanzen die TanzenOP•12/18/23, 3:15 PM

Yeah, I'm really curious to see the results of Dreambooth on a bigger dataset.

�

𝙻𝚎𝚠•12/18/23, 3:15 PM

Guys

�

𝙻𝚎𝚠•12/18/23, 3:15 PM

i found free image generation ai model with no barriers

�

𝙻𝚎𝚠•12/18/23, 3:15 PM

(sometimes)

PPflanzen die Tanzen the idea here is always the same: Describe what should be removable. For Willy ...

mikemenders•12/18/23, 3:56 PM

A familiar trick. So first you create a "character generator" Lora, which is not flexible, then you use the generated images to create a tagged database and train a flexible Lora.

FFurkan Gözükara SECourses i also have auto installer : https://www.patreon.com/posts/92013455

Lorelai AI•12/18/23, 4:15 PM

Thanks I was looking for that

Lorelai AI•12/18/23, 4:37 PM

This might be a dumb question, if you use dreambooth instead of Lora for the training, is the safetensors still used in the Lora folder like a Lora or the checkpoint folder? Or can you do either? And which is best if so?

FFurkan Gözükara SECourses can you tell me what do you see when you open cmd and type python?

goalpwr•12/18/23, 5:07 PM

@Dr. Furkan Gözükara thanks for the quick response, Yes the file was acquired from patreon, my patreon emails is different than my discord email: aljohns

(venv) N:\AI\Stable_Diffusion\ClassImages\SECourses\Deepface>python
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug 1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>

Mmikemenders A familiar trick. So first you create a "character generator" Lora, which is not...

Pflanzen die TanzenOP•12/18/23, 5:35 PM

Yeah. Sort of. It’s all about efficiency. A good base should not take longer than a day including 2 trainings.

vp•12/18/23, 6:12 PM

Udemy also removed access to those that purchased and are offering a refund. Ridiculous.

Pflanzen die TanzenOP•12/18/23, 7:36 PM

Some of you asked for it so I wrote a rather detailed article (warning dry content with no images, just plain detailed insights): https://blib.la/blog/mastering-character-consistency-in-stable-diffusion-sdxl-a-detailed-guide

Blibla | Mastering Character Consistency in Stable Diffusion SDXL

Creating consistent characters in AI art, especially with tools like Stable Diffusion SDXL, requires a blend of creativity, precision, and the right techniques.

pat._.fenis•12/18/23, 7:38 PM

pat._.fenis•12/18/23, 7:39 PM

Tensor core models seem pointless for batch use...

pat._.fenis•12/18/23, 7:40 PM

Not sure what all the hype is.

PPflanzen die Tanzen Some of you asked for it so I wrote a rather detailed article (warning dry conte...

vp•12/18/23, 8:50 PM

Mind adding RSS to your site? Would like to follow the blog.

mikemenders•12/18/23, 9:19 PM

Speaking of keywords, tonight I made a Lora model that I first trained with a keyword that was in the anime models but not (or had little chance of being) in the photo models. It performed relatively well on the photo models, but it confounded the anime models.
I then created a custom keyword from several words and it performed better on both model types.
And one more thing about repetition: I was training poses, and so I didn't use reg images. At first I doubled the number of repetitions, but I got a strong model. After going back to the "100/number of images" formula and using 4 reps instead of 8 (I have 25 images), I got a more flexible model. It was still a bit strong. I trained on Prodigy and 4/4 weights with 0.3 d_coef, I can't go below those values anymore.

mikemenders•12/18/23, 10:06 PM

One more thing about flexibility: I can combine my own loras very well, so I can combine people with poses, styles, and they perform well on several models. A lot of the models that others have made have been so heavily trained that they simply can't be combined, and I'm thinking of generating images with them and re-creating them myself. Plus, some of them are so lousy that it takes tricks to get a good result. I admit I've never been a fan of weighting, because where you have to use a lot of weight, the model doesn't understand what I want. I know anime models love it, but I really find it unnecessary, but sometimes you have to use it for those crappy loras.

LLorelai AI This might be a dumb question, if you use dreambooth instead of Lora for the tra...

Furkan Gözükara SECourses•12/19/23, 12:05 AM

full checkpoints goes into model folder and loras goes into lora folder

Furkan Gözükara SECourses•12/19/23, 12:05 AM

both has .safetensors extension

Ggoalpwr @Dr. Furkan Gözükara thanks for the quick response, Yes the file was acquired f...

Furkan Gözükara SECourses•12/19/23, 12:06 AM

hello. sorry for late reply. have you fixed your issue?

Vvp Udemy also removed access to those that purchased and are offering a refund. Ri...

Furkan Gözükara SECourses•12/19/23, 12:06 AM

yep 100%. i am still waiting answer from them.

Ppat._.fenis Click to see attachment

Furkan Gözükara SECourses•12/19/23, 12:13 AM

i tried only batch size 1 and getting over 70% speed up

FFurkan Gözükara SECourses hello. sorry for late reply. have you fixed your issue?

goalpwr•12/19/23, 1:22 AM

@Dr. Furkan Gözükara No I'm still having the error described above, any advice is greatly appreciated , thanks!

Ggoalpwr @Dr. Furkan Gözükara No I'm still having the error described above, any advice ...

Furkan Gözükara SECourses•12/19/23, 1:25 AM

hello

Jill the girl with short ginger hair and pale skin, a joyful character wearing a yellow sundress and black mary Janes, consistent (character sheet, multiple views, various expressions, character turnaround:1.2)

just a standard LoRA with my usual settings. It is v2. 1. version was trained on 14 images (includi

Similar Threads