Software Engineering Courses (SECourses)•15mo ago

I trained a finetune, and now have a .safetensors model. I am using local Stability matrix to creat

I trained a finetune, and now have a .safetensors model.

I am using local Stability matrix to create images. As What/where do I load the file? VAE? CLIP? Model itself?

druhl•10/14/24, 8:13 AM

Get SwarmUI. Use this tutorial:

https://www.youtube.com/watch?v=HKX8_F1Er_w&t=3273s

YouTubeSECourses

Zero to Hero Stable Diffusion 3 Tutorial with Amazing SwarmUI SD We...

Do not skip any part of this tutorial to master how to use Stable Diffusion 3 (SD3) with the most advanced generative AI open source APP SwarmUI. Automatic1111 SD Web UI or Fooocus are not supporting the #SD3 yet. Therefore, I am starting to make tutorials for SwarmUI as well. #StableSwarmUI is officially developed by the StabilityAI and your m...

druhl•10/14/24, 8:15 AM

Also, this one specifically for SwarmUI x Flux:

https://www.youtube.com/watch?v=bupRePUOA18&t=301s

YouTubeSECourses

FLUX: The First Ever Open Source txt2img Model Truly Beats Midjourn...

FLUX is the first time every an open source txt2img is able to truly surpass and produce better quality and better prompt following images than #Midjourney, Adobe Firefly, Leonardo Ai, Playground Ai, Stable Diffusion, SDXL, SD3 and Dall E3. #FLUX is developed Black Forest Labs and its team is mainly composed by the original authors of #StableDi...

deepblue_1337OP•10/14/24, 8:15 AM

I'll try that as soon as I find the time

deepblue_1337OP•10/14/24, 8:16 AM

another short noob question. I now have a flux model finetuned to a person. I can generate images with that person in it. However as soon as I want another person in there (lets say finetuned person shaking queen elizabeths hand) the finetuned person is not in the image but a random person.

Is that a limiation of the model/finetuning or am I doing something wrong?

druhl•10/14/24, 8:25 AM

Are you using unique token? Like ohwx man shaking queen elizabeth's hand?

deepblue_1337OP•10/14/24, 8:26 AM

yes

deepblue_1337OP•10/14/24, 8:27 AM

thats actually exactly the input I was using (with my token)

druhl•10/14/24, 9:32 AM

lol is that a dreambooth thing? Lol, not sure. Try with more checkpoints.

druhl•10/14/24, 9:33 AM

When using the character Lora, the problem is never that it does not execute, rather it executes on all the people

druhl•10/14/24, 9:34 AM

You'll get a queen elizabeth version of yourself presenting the award to yourself

Ddruhl Hi, Doc. There is a typo error in Line-4: "Fine Tuning / DreamBooth: 256 Trainin...

Furkan Gözükara SECourses•10/14/24, 9:49 AM

ah yes will fix now ty

VVigilence @Dr. Furkan Gözükara Have you tested flux dora vs lora extracted from finetune y...

Furkan Gözükara SECourses•10/14/24, 9:49 AM

i didnt do any dora yet kohya still dont have

Ddeepblue_1337 I was following the tutorial and ran a finetune on massed compute, unfortunately...

Furkan Gözükara SECourses•10/14/24, 9:57 AM

you didnt . it has a limited space you need to save fewer checkpoints

Furkan Gözükara SECourses•10/14/24, 9:57 AM

or upload checkpoints and delete

DDarth Teethius Has anyone here done full flux on a 3060?

Furkan Gözükara SECourses•10/14/24, 9:57 AM

yes i did

Furkan Gözükara SECourses•10/14/24, 9:57 AM

it is 40 second / it

Furkan Gözükara SECourses•10/14/24, 9:58 AM

requires min 48 gb ram

Ddeepblue_1337 I trained a finetune, and now have a .safetensors model. I am using local Stabi...

Furkan Gözükara SECourses•10/14/24, 9:58 AM

i dont know there but in swarmui it is diffusion_models folder

Ddeepblue_1337 another short noob question. I now have a flux model finetuned to a person. I ca...

Furkan Gözükara SECourses•10/14/24, 9:58 AM

yes it is bleeding

Furkan Gözükara SECourses•10/14/24, 9:58 AM

currently we couldnt fix

Furkan Gözükara SECourses•10/14/24, 9:58 AM

there are some attempts being worked on

Furkan Gözükara SECourses•10/14/24, 9:58 AM

that can fix this

Furkan Gözükara SECourses•10/14/24, 9:59 AM

this is biggest problem of flux

Furkan Gözükara SECourses•10/14/24, 10:02 AM

expression test grids added

FFurkan Gözükara SECourses this is biggest problem of flux

druhl•10/14/24, 10:05 AM

I also think pose training does not work with Flux

druhl•10/14/24, 10:06 AM

I've rented multiple GPUs since last two days and tried dreambooth training with various concepts. None have worked.

druhl•10/14/24, 10:08 AM

I had surmised earlier that maybe flux_dev fails to learn multiple poses at the same time, so I tried training flux_dev_de_distill, but that did not work either.

druhl•10/14/24, 10:09 AM

Right now, I am lora training a single pose, E.g., 'ohwx pose'. 15 images x 320 epochs (Rank1 settings & all images captioned as ohwx pose). If that does not work either, I'm not sure what will.

druhl•10/14/24, 10:14 AM

I guess Controlnets are still king when it comes to this sort of thing.

But they don't seem to be as powerful with Flux as they were with SD.

Ddruhl I guess Controlnets are still king when it comes to this sort of thing. 🙂 But t...

Furkan Gözükara SECourses•10/14/24, 10:15 AM

very likely

Furkan Gözükara SECourses•10/14/24, 10:15 AM

FFurkan Gözükara SECourses Click to see attachment

druhl•10/14/24, 10:19 AM

Undoutedly, the best character dreambooth workflow out there atm.

FFurkan Gözükara SECourses Click to see attachment

druhl•10/14/24, 10:20 AM

Doc, had a question regarding dataset preparation! What if one artificially manipulated the background on the dataset to make it more varied (E.g., using photoshop or lightroom? Will the trainining pick up edge artifacts?

Ddruhl Doc, had a question regarding dataset preparation! What if one artificially mani...

Furkan Gözükara SECourses•10/14/24, 10:21 AM

yep very likely

Furkan Gözükara SECourses•10/14/24, 10:21 AM

ai picks those :d

druhl•10/14/24, 10:23 AM

Haha, thought so. I've been very careful since that one time it picked up a mosquito lol

druhl•10/14/24, 10:23 AM

Ddruhl Haha, thought so. I've been very careful since that one time it picked up a mosq...

Furkan Gözükara SECourses•10/14/24, 10:26 AM

yep

FFurkan Gözükara SECourses Click to see attachment