Software Engineering Courses (SECourses)•15mo ago

it says same process as lora, only different config. Do I still use the LoRa tab in the webinterface

it says same process as lora, only different config. Do I still use the LoRa tab in the webinterface or do I use dreambooth?

Thanks
Daniel

Ddeepblue_1337 it says same process as lora, only different config. Do I still use the LoRa tab...

druhl•10/12/24, 7:59 PM

Do dreambooth

deepblue_1337OP•10/12/24, 8:01 PM

I have a RTX 3060 and am getting ~45sec/it, which would take quite long for 9200 steps, is that normal?

deepblue_1337OP•10/12/24, 8:02 PM

nvitop is showing >90% GPU utilization

Ddsienra @Dr. Furkan Gözükara flux-dev-de-distill what I saw was that at 30 epoch's the r...

Furkan Gözükara SECourses•10/12/24, 9:56 PM

nice test

Ddeepblue_1337 it says same process as lora, only different config. Do I still use the LoRa tab...

Furkan Gözükara SECourses•10/12/24, 9:57 PM

dreambooth tab

Ddeepblue_1337 I have a RTX 3060 and am getting ~45sec/it, which would take quite long for 9200...

Furkan Gözükara SECourses•10/12/24, 9:57 PM

yes normal

Furkan Gözükara SECourses•10/12/24, 9:57 PM

rtx 3060 takes around 40 second / it in long run

FFurkan Gözükara SECourses nice test

dsienra•10/12/24, 10:04 PM

The training was finished, resemblance is almost perfect, but it still looks a little undertrained, with regularization images it needs much more steps or increase the learning rate to get resemblance. I will resume the training from 160 repetitions per image and I will report back, there was no bleeding between the trained subjects

Ddsienra The training was finished, resemblance is almost perfect, but it still looks a l...

Furkan Gözükara SECourses•10/12/24, 10:05 PM

wow awesome

Furkan Gözükara SECourses•10/12/24, 10:06 PM

i am waiting that model to become more mature

Furkan Gözükara SECourses•10/12/24, 10:06 PM

will fully test it and compare

FFurkan Gözükara SECourses will fully test it and compare

dsienra•10/12/24, 10:27 PM

Nice, If you have to train more than one subject it is your only choice, in my tests is superior on everything, the only problem is if you are going to do a full training the inference of the resulting model will be slower because you have to set the cfg scale to 3.5, to overcome it what can be done is to do the finetune, extract the lora and then use the resulting lora on regular flux-dev and be able to use cfg scale 1 in conjunction with distilled cfg

Ddsienra Nice, If you have to train more than one subject it is your only choice, in my t...

Furkan Gözükara SECourses•10/12/24, 10:30 PM

true

Furkan Gözükara SECourses•10/12/24, 10:30 PM

once you use cfg > 1 = it doubles the duration of generation

FFurkan Gözükara SECourses once you use cfg > 1 = it doubles the duration of generation

dsienra•10/12/24, 10:36 PM

the quality of an extracted lora from a finetune is comparable to a finetune?

Ddsienra the quality of an extracted lora from a finetune is comparable to a finetune?

Furkan Gözükara SECourses•10/12/24, 10:37 PM

almost

Furkan Gözükara SECourses•10/12/24, 10:37 PM

i made a comparison

Furkan Gözükara SECourses•10/12/24, 10:37 PM

but i doubt it would work on

Furkan Gözükara SECourses•10/12/24, 10:37 PM

dev model

FFurkan Gözükara SECourses dev model

dsienra•10/12/24, 10:39 PM

the loras I'm training on flux-dev-de-distill works perfect on regular flux-dev, why the extracted lora will not work?

dsienra•10/12/24, 10:41 PM

not in the other way around the loras trained on regular flux-dev lost resemblance on flux-dev-de-distill on inference

Ddsienra the loras I'm training on flux-dev-de-distill works perfect on regular flux-dev,...

Furkan Gözükara SECourses•10/12/24, 10:42 PM

wow if working then yes

Furkan Gözükara SECourses•10/12/24, 10:42 PM

should work

Ddsienra the loras I'm training on flux-dev-de-distill works perfect on regular flux-dev,...

Furkan Gözükara SECourses•10/12/24, 10:42 PM

have you compared grid side by side?

dsienra•10/12/24, 10:43 PM

no I just created some pictures to test

dsienra•10/12/24, 10:47 PM

I resumed the training from the last lora checkpoint

dsienra•10/12/24, 10:55 PM

@Dr. Furkan Gözükara on flux-dev-de-distill to get resemblance with regularization I used a much higher lr 0.004 on 160 epochs looks still a little undertrained, my lora rank and alpha is set like this, this can be a problem

Ddsienra @Dr. Furkan Gözükara on flux-dev-de-distill to get resemblance with regularizati...

Furkan Gözükara SECourses•10/12/24, 10:55 PM

yep

Furkan Gözükara SECourses•10/12/24, 10:55 PM

you basically changed the LR i researched

Furkan Gözükara SECourses•10/12/24, 10:55 PM

reduced it like 1/8

Furkan Gözükara SECourses•10/12/24, 10:55 PM

i am offline

FFurkan Gözükara SECourses yep

dsienra•10/12/24, 10:57 PM

ohhhh, now I understands

dsienra•10/12/24, 11:01 PM

No wonder it was taking me so long to train my LoRAs, the learning rate was too low

dsienra•10/12/24, 11:16 PM

@Dr. Furkan Gözükara sorry that you are offline, no hurry to answer, in your best configs nowhere says if the config is a finetune or a lora

Ddsienra @Dr. Furkan Gözükara sorry that you are offline, no hurry to answer, in your bes...

Error-404-unknown•10/13/24, 12:08 AM

LoRas are in Best_config folder inside kohya_gui_flux_installer.zip and finetunes are in Best_config_v5.zip

Ddsienra the loras I'm training on flux-dev-de-distill works perfect on regular flux-dev,...

hamuz•10/13/24, 12:12 AM

It's great to hear that! I just started training a LoRA about an hour ago, but I canceled it now because I'm going to rent 2 A100s for the training. Do you have any parameters that you recommend adjusting between De-Distilled and Standard flux LoRA training?

Hhamuz It's great to hear that! I just started training a LoRA about an hour ago, but I...

dsienra•10/13/24, 12:25 AM

you can use the same settings, the samples during training doesn't work, other than that the same, you can train multiple subjects in one lora with the de-distilled model and learn new concepts much better

EError-404-unknown LoRas are in Best_config folder inside kohya_gui_flux_installer.zip and finetune...

dsienra•10/13/24, 12:26 AM

thanks!!

Ddsienra you can use the same settings, the samples during training doesn't work, other t...

hamuz•10/13/24, 12:42 AM

Thank you so much

Ddsienra The training was finished, resemblance is almost perfect, but it still looks a l...

druhl•10/13/24, 1:32 AM

Can you resume/ increase the no. of epochs in the existing training? Or is it that you had paused it?

Ddsienra @Dr. Furkan Gözükara on flux-dev-de-distill to get resemblance with regularizati...

druhl•10/13/24, 1:41 AM

Isn't 0.004 faster than 0.000004? Or is LR counter opposite in ML?

druhl•10/13/24, 4:22 AM

Sadly, my pose training has completely failed on both the models. Think I'm doing something wrong on the fundamental level itself

, 'cuz I'm approaching it like a character fine-tune.

Ddsienra Nice, If you have to train more than one subject it is your only choice, in my t...

druhl•10/13/24, 5:28 AM

Get what you're trying to say. Changing strategy, training different poses as multiple separate concepts instead of training them under the same poses class.

druhl•10/13/24, 5:29 AM

Inference on dev_distill can be pretty good as well, if you tweak it in Comfy. But yes, it is slower.

druhl•10/13/24, 5:31 AM

flux_dev versus flux_dev_de_distill

druhl•10/13/24, 5:32 AM

versus Pro 1.1

Ddruhl Can you resume/ increase the no. of epochs in the existing training? Or is it th...

dsienra•10/13/24, 5:51 AM

you can get any lora checkpoint and resume the training from there, there is no pause option.

Ddruhl Isn't 0.004 faster than 0.000004? Or is LR counter opposite in ML?

dsienra•10/13/24, 6:02 AM

0.0004 is much faster but it was my mistake, my lora rank was at 128 and my network alpha at 16 that means that I had to increase the lr up to 0.0004 to match the dr. Furkan settings, just use one of his lora configs and you are good to go. lora rank 128 alpha 1 or rank 128 alpha 128 is the same setting, because 128 divided 128 equals 1, all this iinfo is kohya's documentation

Ddruhl Get what you're trying to say. Changing strategy, training different poses as mu...

dsienra•10/13/24, 6:19 AM

unless the poses you are training are very uncommon may be the pose is already trained on the base model, you have to prompt it correctly, if the pose is kind of unique then you can have a dataset of pictures with your especial pose and train it with captions like "nameofthepose pose"

Ddsienra unless the poses you are training are very uncommon may be the pose is already t...

druhl•10/13/24, 6:22 AM

That's how I'm captioning my images right now to start new training. But what about unique token and class token? Do I need to enter all of them comma seperated or I can just go into the kohya directory directly & modify folder names like 1_nameofthefirstpose, 1_nameofthesecondpose, and so on ?

it says same process as lora, only different config. Do I still use the LoRa tab in the webinterface

Similar Threads