Software Engineering Courses (SECourses)•15mo ago

<@205854764540362752> just in case you are thinking about training TE - because that would make sens

@Furkan Gözükara SECourses just in case you are thinking about training TE - because that would make sense with regularization, to separate the concepts further - the code isn't ready for that. it only unhooks the lora from the transformer.

Ddxqb <@205854764540362752> just in case you are thinking about training TE - because ...

Furkan Gözükara SECourses•10/5/24, 6:02 PM

well it doesnt matter. i tested all cases all bleeds huge

Furkan Gözükara SECourses•10/5/24, 6:02 PM

and it is issue of the dit architecture

Furkan Gözükara SECourses•10/5/24, 6:02 PM

not TE or clip training

Furkan Gözükara SECourses•10/5/24, 6:02 PM

so i will train only unet to test

dxqbOP•10/5/24, 6:03 PM

what I mean is: if this reg method works well, it might even be working better with TE training. but not yet, so turn it off either way

Furkan Gözükara SECourses•10/5/24, 6:04 PM

Maxivi•10/5/24, 6:05 PM

@Furkan Gözükara SECourses seems prior preservation in kohya is broken

Maxivi•10/5/24, 6:06 PM

https://github.com/kohya-ss/sd-scripts/pull/1374#issuecomment-2395126424

GitHub

support SD3 by kohya-ss · Pull Request #1374 · kohya-ss/sd-scripts

Replace SD3Tokenizer with the original CLIP-L/G/T5 tokenizers.
Extend the max token length to 256 for T5XXL.
Refactor caching for latents.
Refactor caching for Text Encoder outputs
Extract arch...

Maxivi•10/5/24, 6:07 PM

bghira responded

Furkan Gözükara SECourses•10/5/24, 6:08 PM

i think he didnt see my reply

becaue i added. but good

Furkan Gözükara SECourses•10/5/24, 6:08 PM

if this get fixed nice

FFurkan Gözükara SECourses kk

dxqbOP•10/5/24, 6:23 PM

ah, and batch size 1!

Ddxqb ah, and batch size 1!

Furkan Gözükara SECourses•10/5/24, 6:24 PM

batch size 1 is best

Furkan Gözükara SECourses•10/5/24, 6:24 PM

anyway for low number of images

Ddxqb <@205854764540362752> Flux anti-bleeding protection. look at right-most column....

dxqbOP•10/6/24, 3:13 AM

with and without
[removed]

Ddxqb with and without [removed]

crystalwizard•10/6/24, 3:15 AM

hilary doesn't look like hilary

Ccrystalwizard hilary doesn't look like hilary

dxqbOP•10/6/24, 3:16 AM

that's the base model, I didn't train Clinton. The point is that Clinton stays as Clinton as she was

dxqbOP•10/6/24, 3:16 AM

on the left

Ddxqb that's the base model, I didn't train Clinton. The point is that Clinton stays a...

crystalwizard•10/6/24, 3:16 AM

she didn't though. neither woman, in both images, look like they do on the right

dxqbOP•10/6/24, 3:16 AM

the trained person also looks a bit worse but that can probably be tweaked

dxqbOP•10/6/24, 3:16 AM

yeah

Ddxqb the trained person also looks a bit worse but that can probably be tweaked

crystalwizard•10/6/24, 3:17 AM

they don't just look worse, they look like different people

crystalwizard•10/6/24, 3:17 AM

what if you try famous comic book charcters like spider man or deadpool

dxqbOP•10/6/24, 3:18 AM

and then?

crystalwizard•10/6/24, 3:18 AM

see how they come out with and without your anti-bleed

dxqbOP•10/6/24, 3:20 AM

the base model will have no problem differentiating a woman from spiderman. even woman and man is easy. on two woman or two men have this bleeding effect

crystalwizard•10/6/24, 3:21 AM

sure, and they're costumes, but the thought is - is this only affecting human faces or is it going to affect other things?

Ccrystalwizard she didn't though. neither woman, in both images, look like they do on the right

dxqbOP•10/6/24, 3:28 AM

I think you might have misunderstood the samples. Here is what you've probably meant, untrained vs. trained

dxqbOP•10/6/24, 3:28 AM

the training of the left person obviously isn't good, but Clinton stays mostly unchanged

dxqbOP•10/6/24, 3:33 AM

removed above to avoid further misunderstanding

Maxivi•10/6/24, 9:51 AM

@Furkan Gözükara SECourses this is what kohya said about @dxqb method

MMaxivi <@205854764540362752> this is what kohya said about <@1236274587520598136> metho...

dxqbOP•10/6/24, 10:29 AM

He is right, it is very similar. You could have exactly the same effect by pregenerating and therefore make this possible for full finetune with no additional vram, too. You'd have to synchronize timesteps and seed(!) between the pregenerated data and the training though, which is the difference to the current "Prior Preservation" feature. I did implement this for full finetune also, but without pregeneration so it needs a lot of VRAM for two full models, the student and the trainer model. Feel free to forward my contact.

dxqbOP•10/6/24, 10:39 AM

On the second comment though, it's not only activation. In this sample, dreambooth-training, Clinton would look like the left person without regularization.

dxqbOP•10/6/24, 10:39 AM

the left person in the right sample, I mean

MMaxivi <@205854764540362752> this is what kohya said about <@1236274587520598136> metho...

Furkan Gözükara SECourses•10/6/24, 11:39 AM

my training completed now will test

Furkan Gözükara SECourses•10/6/24, 11:48 AM

OneTrainer strategy looks like failed for me