Software Engineering Courses (SECourses)•15mo ago

I am trying to do a full Dreambooth finetune on massed compute with 2x A6000. I am using the config

I am trying to do a full Dreambooth finetune on massed compute with 2x A6000. I am using the config provided on patreon for 48gb gpu. I also set necessary parameters in accelerate tab but it keeps crashing due to OOM.

With one A6000 it worked just fine. Also is bucketing currently supported for Flux Dreambooth?

Ddruhl @Dr. Furkan Gözükara how does merging loras work? Any resources for learning tha...

Furkan Gözükara SECourses•10/14/24, 12:11 PM

it is purely experimental

druhl•10/14/24, 12:11 PM

I see

EElwii I am trying to do a full Dreambooth finetune on massed compute with 2x A6000. I ...

Furkan Gözükara SECourses•10/14/24, 12:11 PM

dreambooth requires 80 gb

Furkan Gözükara SECourses•10/14/24, 12:11 PM

each gpu

Furkan Gözükara SECourses•10/14/24, 12:11 PM

you can do only lora

FFurkan Gözükara SECourses dreambooth requires 80 gb

ElwiiOP•10/14/24, 12:12 PM

I am pretty sure I did a full Finetune with 1x A6000 the other day

ElwiiOP•10/14/24, 12:13 PM

And the model came out pretty good, full 23GB checkpoints

EElwii I am pretty sure I did a full Finetune with 1x A6000 the other day😅

Furkan Gözükara SECourses•10/14/24, 12:14 PM

1x uses 27.5 gb

Furkan Gözükara SECourses•10/14/24, 12:14 PM

2x uses 77.5

Furkan Gözükara SECourses•10/14/24, 12:14 PM

because optimizations not working

Furkan Gözükara SECourses•10/14/24, 12:15 PM

i am talking with kohya he couldnt solve yet

ElwiiOP•10/14/24, 12:15 PM

Oh alright I see, so just 1 A6000 gpu is working for dreambooth currently

EElwii Oh alright I see, so just 1 A6000 gpu is working for dreambooth currently

Furkan Gözükara SECourses•10/14/24, 12:15 PM

yep

Furkan Gözükara SECourses•10/14/24, 12:15 PM

it works even for 6 gb gpus :d

Furkan Gözükara SECourses•10/14/24, 12:16 PM

i made a very detailed publish just few minutes ago

SiggiSmilez•10/14/24, 12:16 PM

How much does a full fine tune cost on mass compute/runpod?

Furkan Gözükara SECourses•10/14/24, 12:16 PM

i think read it entirely

SSiggiSmilez How much does a full fine tune cost on mass compute/runpod?

Furkan Gözükara SECourses•10/14/24, 12:16 PM

published it

SiggiSmilez•10/14/24, 12:16 PM

Oh thanks

Furkan Gözükara SECourses•10/14/24, 12:16 PM

please read entirely : https://www.patreon.com/posts/kohya-flux-fine-112099700

Patreon

Kohya FLUX Fine Tuning (Full Checkpoints) Training Full Tutorial Fo...

Get more from SECourses: FLUX, Tutorials, Guides, Resources, Training, Scripts on Patreon

SSiggiSmilez How much does a full fine tune cost on mass compute/runpod?

ElwiiOP•10/14/24, 12:16 PM

I think I paid around 3 - 4$

Furkan Gözükara SECourses•10/14/24, 12:16 PM

this is a gold post took me more than 1 week

Furkan Gözükara SECourses•10/14/24, 12:16 PM

as low as 1 $ depends on dataset

FFurkan Gözükara SECourses this is a gold post took me more than 1 week

SiggiSmilez•10/14/24, 12:17 PM

Means that I have to be a gold member to see the post? Sorry I am entirely new to Patreon

SiggiSmilez•10/14/24, 12:41 PM

Oh, I see, I can see the text but I can't access the links

SSiggiSmilez Means that I have to be a gold member to see the post? Sorry I am entirely new t...

ElwiiOP•10/14/24, 12:45 PM

You can access it even with lowest supporter tier

SSiggiSmilez Means that I have to be a gold member to see the post? Sorry I am entirely new t...

Furkan Gözükara SECourses•10/14/24, 12:49 PM

no all members can see and access

SiggiSmilez•10/14/24, 1:51 PM

When I do a fine tune, I can download the checkpoint and use it in comfyui, right?

SSiggiSmilez When I do a fine tune, I can download the checkpoint and use it in comfyui, righ...

Furkan Gözükara SECourses•10/14/24, 1:53 PM

100%

SiggiSmilez•10/14/24, 1:54 PM

But the checkpoint size is around the same size as flux?

mayna•10/14/24, 2:02 PM

Hello community, I am having a 2gb lora on SD , it was like 147mo only for 40mn ,
what I am doing wrong now ? the file is bigger and takes about 3h , the lora does not even look alike

heeeelp

SSiggiSmilez But the checkpoint size is around the same size as flux?

Furkan Gözükara SECourses•10/14/24, 2:08 PM

yes

Furkan Gözükara SECourses•10/14/24, 2:08 PM

23.8 gb

Mmayna Hello community, I am having a 2gb lora on SD , it was like 147mo only for 4...

Furkan Gözükara SECourses•10/14/24, 2:09 PM

lora network dimension

Furkan Gözükara SECourses•10/14/24, 2:09 PM

increases size

mayna•10/14/24, 2:10 PM

I used dreambooth training ( not lora only ) Tab 1 not 2 , what should I use ( for face only lora )

mayna•10/14/24, 2:15 PM

it worked you are the best @Dr. Furkan Gözükara

dsienra•10/14/24, 3:16 PM

@Dr. Furkan Gözükara Update on my https://huggingface.co/nyanko7/flux-dev-de-distill training tests. Some bad news and some good news, Good news: I can train many subjects at the same time without bleeding between each other 11 people in one LORA, simple captions "Name class" note: the token must be different a saw a little bleeding with similar names like "Diego man" "Dani man" works best with "NameLastname class" so they end up been very different. Bad news: there is still some class bleeding, may be my fault because I was using a higher lr than recommended to get faster results, Other thing I was using faster presets rather than quality presets "Apply T5 Attention Mask disabled" "Train T5-XXL disabled" now I'm testing with dose enabled, other bad news regularization images still reduces resemblance, I will try this again with "Apply T5 Attention Mask enabled" and "Train T5-XXL enabled" and with the recommended learning rate. The other option that I didn't try was "Guidance scale for Flux1" flux-dev-de-distill has a default cfg of 3.5, that option I leave it at 1 for now because I'm using regular Flux-Dev for inference is another thing I have to test. I will report my Updates.

nyanko7/flux-dev-de-distill · Hugging Face

Ddsienra @Dr. Furkan Gözükara Update on my https://huggingface.co/nyanko7/flux-dev-de-dis...

Furkan Gözükara SECourses•10/14/24, 3:18 PM

awesome

Furkan Gözükara SECourses•10/14/24, 3:18 PM

hopefully it will be my next research after i make tutorial

Furkan Gözükara SECourses•10/14/24, 3:48 PM

public article : https://www.patreon.com/posts/113970485

Patreon

Huge FLUX LoRA vs Fine Tuning / DreamBooth Experiments Completed, M...

Get more from SECourses: FLUX, Tutorials, Guides, Resources, Training, Scripts on Patreon

Ddsienra @Dr. Furkan Gözükara Update on my https://huggingface.co/nyanko7/flux-dev-de-dis...

deepblue_1337•10/14/24, 4:03 PM

So you add the image captions, is that correct? Literally "NameLastname class"? so "JohnDoe class"?

Ddeepblue_1337 So you add the image captions, is that correct? Literally "NameLastname class"? ...

dsienra•10/14/24, 4:44 PM

yes thats it

dsienra•10/14/24, 4:46 PM

"NameLastname man" "NameLastname woman" etc.

SiggiSmilez•10/14/24, 5:16 PM

I have questions about Flux Lora training especially about prompting for characters:

SiggiSmilez•10/14/24, 5:16 PM

Do you prompt at all?
Do you use tags or natural language?

SiggiSmilez•10/14/24, 5:17 PM

What do you describe in the prompt?
Everything that is important about the character or everything that is not important about the character?
Or everything in the training picture?

SSiggiSmilez I have questions about Flux Lora training especially about prompting for charact...

dsienra•10/14/24, 5:21 PM

I use "Charactername class and the normal prompt" or "Charactername class, prompt"

dsienra•10/14/24, 5:22 PM

in the prompt put all you want, the clothing the setting location what is doing, etc

Ddsienra I use "Charactername class and the normal prompt" or "Charactername class, promp...

Sipriyani•10/14/24, 5:26 PM

are you using kohya? if so how do you feel these boxes:

Insance prompt: diego man,
and class prmpt: man .

I am trying to do a full Dreambooth finetune on massed compute with 2x A6000. I am using the config

Similar Threads

Similar Threads

Similar Threads