Software Engineering Courses (SECourses)•15mo ago

Hello I attempted Flux Dreambooth Finetune but ran out of RAM. It filled up and then crashed saying

Hello I attempted Flux Dreambooth Finetune but ran out of RAM. It filled up and then crashed saying out of memory. I have seen this issue in the channel a few times already but never found a solution. (My System: RTX 3060 12GB, 32 GB RAM)

On the Patreon post @Dr. Furkan Gözükara said: Set Virtual RAM to 64 GB + Enable Shared VRAM

What exactly do you need to do? A more precise description would be great

Ddsienra this a straight LORA training, this is my first test with the model flux-dev-de-...

wardensc2•10/9/24, 2:50 PM

Do you use regulation images folder to train or use caption or don't use anything ?

EElwii Hello I attempted Flux Dreambooth Finetune but ran out of RAM. It filled up and ...

dsienra•10/9/24, 2:52 PM

dsienra•10/9/24, 2:52 PM

there you set you virtual memory size

dsienra•10/9/24, 2:52 PM

64 GB in you case

Ddsienra Click to see attachment

ElwiiOP•10/9/24, 2:54 PM

Thanks, I already set maximum to 64GB, for the initial I used the old on at around 5GB but this did not resolve the issue

Wwardensc2 Do you use regulation images folder to train or use caption or don't use anythin...

dsienra•10/9/24, 3:04 PM

I did my first test, very promising results, a little undertrained at the moment, at the end I resumed the training with regularization images still no bleeding, now I going to start a new training from scratch 11 people at the same time, family and friends, I going to use regularization images from the class person because I'm training people form different ages and genders the regularization images are captioned I downloaded the dataset including captions a time ago it contains very diverse people from different races and ages from baby to elderly people, male and female. the captions of my subjects are simple "name class" lets see how it behaves

EElwii Thanks, I already set maximum to 64GB, for the initial I used the old on at arou...

dsienra•10/9/24, 3:06 PM

do you have enough free space on the selected disk? you are using the correct preset for your system?

Ddsienra do you have enough free space on the selected disk? you are using the correct pr...

ElwiiOP•10/9/24, 3:09 PM

Yes I have half a TB free space on the M.2 ssd, I downloaded Best_Configs_v5.zip and used the 12GB one (12GB_GPU_10800MB_17.2_second_it_Tier_1.json)

dsienra•10/9/24, 3:11 PM

idk, it should work, lets wait for Dr. Furkan may be he can help

Ddsienra do you have enough free space on the selected disk? you are using the correct pr...

ElwiiOP•10/9/24, 3:11 PM

In task manager my gpu settings say 16Gb total shared gpu-storage, is that correct?

Ddsienra idk, it should work, lets wait for Dr. Furkan may be he can help

ElwiiOP•10/9/24, 3:11 PM

Yes alright thank you

dsienra•10/9/24, 3:12 PM

you welcome

Ddsienra I did my first test, very promising results, a little undertrained at the moment...

wardensc2•10/9/24, 3:19 PM

I try to train myself but before that I need more info. Normally when I train a lora of one person if the background have some people behind, somehow the image of the person I train bleed to the background into the people. So in your theory, by training with open dev flux checkpoint we can eliminate this problem right ?

Vigilence•10/9/24, 3:36 PM

@Dr. Furkan Gözükara Is it possible to train a flux lora or flux finetune from a flux q8 gguf format?

Wwardensc2 I try to train myself but before that I need more info. Normally when I train a ...

dsienra•10/9/24, 3:37 PM

The most important thing and the main difference with this model is that I can train multiple people of the same class without bleeding, with regular flux dev I I trained 3 man for example and the model mixed all the faces together, and on inference prompting for one person returns a mix of all, now the problem is gone with flux-dev-de-distill, it behaves like sdxl but with much better quality. I still experimenting but is very promising so far.

Ddsienra The most important thing and the main difference with this model is that I can t...

wardensc2•10/9/24, 3:43 PM

So that mean 1 lora 3 lora active keyword is possible but can they appear all together in 1 image ? You need to check it

Wwardensc2 So that mean 1 lora 3 lora active keyword is possible but can they appear all to...

dsienra•10/9/24, 3:45 PM

I will check that, give me a minute, I trained 1 lora 11 people

wardensc2•10/9/24, 3:46 PM

thank you so much

Wwardensc2 So that mean 1 lora 3 lora active keyword is possible but can they appear all to...

dsienra•10/9/24, 4:29 PM

One at a time works perfect, two or more people on the same frame loose likeness, can be fixed with inpainting, man and woman works much better, the same problem that sdxl have, but remember my lora is still undertrained, lets see with my new lora... I will start training later, it will take 2 days or more to finish

Vigilence•10/9/24, 4:29 PM

Comparison of all quants. Q8 is clearly the winner, when compared to fp16.

daroth•10/9/24, 4:31 PM

"clearly". sample size of 1

Vigilence•10/9/24, 4:33 PM

I have seen other posts with comparison. Do you have a different experience?

daroth•10/9/24, 4:36 PM

I've made no comparisons. I just use fp8 to generate

Vigilence•10/9/24, 4:38 PM

From the comparison posts on reddit, q8 seems to keep a similar consistency to fp16. So if we could use it for training that would be super. But that doesn't seem possible atm.

Vigilence•10/9/24, 4:41 PM

@daroth Have you finetuned using fp8?

daroth•10/9/24, 4:44 PM

no, kohya didn't load that model. only the full sized one worked

wardensc2•10/9/24, 4:45 PM

fp8 is faster when you have rtx 4000s

OinkSauce•10/9/24, 4:45 PM

If anyone has a second... Could i get more information on what "de-distilled" means?

wardensc2•10/9/24, 4:45 PM

q8 faster in other cases

OOinkSauce If anyone has a second... Could i get more information on what "de-distilled" m...

wardensc2•10/9/24, 4:46 PM

read in this: https://huggingface.co/nyanko7/flux-dev-de-distill

nyanko7/flux-dev-de-distill · Hugging Face

Ddsienra One at a time works perfect, two or more people on the same frame loose likeness...

wardensc2•10/9/24, 4:48 PM

If your work is success, I will train a checkpoint with maybe 12 people inside it, maybe quality much better than use 12 loras

Vigilence•10/9/24, 4:48 PM

I'm looking to do a finetune with a fp8 but the Dr. setting for it uses shared memory, and is slow at around 10s/it. But speaking to koyha, it seems its an intended behavior to use shared memory.

VVigilence I'm looking to do a finetune with a fp8 but the Dr. setting for it uses shared m...

wardensc2•10/9/24, 4:49 PM

just learn to use masscompute service or runpod. Local VGA is not mean to change model anymore. Quality is worst.

Vigilence•10/9/24, 4:49 PM

I might have to, but I can do super fast loras, so I was suprised how slow it was.

wardensc2•10/9/24, 4:50 PM

I use 3090 with 24gb Vram but I can only train fp8 lora. I use mass compute to train fully fp16 lora with the cost of about 4 usd

wardensc2•10/9/24, 4:50 PM

very cheap

Vigilence•10/9/24, 4:50 PM

That is per hour?

wardensc2•10/9/24, 4:50 PM

no, total cost

OinkSauce•10/9/24, 4:51 PM

Massed Compute - from my experience has been great, far better than runpod. The SECourses pricing on the A6000's is great.

wardensc2•10/9/24, 4:51 PM

per hour is only 1.25 usd

wardensc2•10/9/24, 4:52 PM

when training with my local pc, I need 11 hours of 3090 run full throttle, my poor pc fan roar the hell of of my room and the temperature turn up a lot

wardensc2•10/9/24, 4:52 PM

so 4 usd saving my trouble and time

wardensc2•10/9/24, 4:53 PM

11 hours of 3090 for fp8 lora

wardensc2•10/9/24, 4:53 PM

no thank you

OOinkSauce If anyone has a second... Could i get more information on what "de-distilled" m...

dsienra•10/9/24, 4:53 PM

I'm training this model now, Works much better for training, you can train multiple people on one lora without concept mixing, it behaves very much like sdxl, much more flexible

Vigilence•10/9/24, 4:53 PM

Yeah the pc lights stayed on and it looks like a disco ballroom.

Ddsienra I'm training this model now, Works much better for training, you can train multi...

wardensc2•10/9/24, 4:54 PM

I never see any loras in SDXL have multiple concept, Flux maybe the first model can do it

dsienra•10/9/24, 4:55 PM

I trained loras on sdxl with more than 20 people no problem

wardensc2•10/9/24, 4:56 PM

Since SDXL have instance ID, so I use it for more convenience if I need people lora anyway

Vigilence•10/9/24, 4:57 PM

Best gpu prices are from here - https://www.autodl.com/home but I don't know how to run them. Learned about them via an article discussing 48gb 4090s.

AutoDL算力云 | 弹性、好用、省钱。租GPU就上AutoDL

AutoDL为您提供专业的GPU租用服务，秒级计费、稳定好用，高规格机房，7x24小时服务。更有算法复现社区，一键复现算法。

Hello I attempted Flux Dreambooth Finetune but ran out of RAM. It filled up and then crashed saying

Similar Threads

Similar Threads

Similar Threads