Software Engineering Courses (SECourses)•16mo ago

has anyone got the only training specific layers yet to see a speed increase? im getting same speed

has anyone got the only training specific layers yet to see a speed increase? im getting same speed trying to training only 2 layers as i do when i train all

DDarth Teethius has anyone got the only training specific layers yet to see a speed increase? im...

xlads•9/17/24, 8:52 PM

what are layers?

Xxlads I’m confused on what fine tuning is, do we basically do the same training as in ...

Furkan Gözükara SECourses•9/17/24, 10:07 PM

yes true

Furkan Gözükara SECourses•9/17/24, 10:07 PM

exactly same

Furkan Gözükara SECourses•9/17/24, 10:07 PM

just use new config

Furkan Gözükara SECourses•9/17/24, 10:08 PM

checkpoints will be 23.8 gb

Furkan Gözükara SECourses•9/17/24, 10:08 PM

load config into dreambooth tab

RRa25 I did a finetune of flux dev, results are amazing. I use Extract Flux LoRA tab t...

Furkan Gözükara SECourses•9/17/24, 10:08 PM

i havent could be broken

Furkan Gözükara SECourses•9/17/24, 10:08 PM

if broken should be reported

Furkan Gözükara SECourses•9/17/24, 10:09 PM

i will try extract now

Furkan Gözükara SECourses•9/17/24, 10:20 PM

ok i think i got the best single layer found

Furkan Gözükara SECourses•9/17/24, 10:20 PM

time to deepen research

Furkan Gözükara SECourses•9/17/24, 11:17 PM

it sucks that fused back pass not works with multi gpu

Furkan Gözükara SECourses•9/17/24, 11:17 PM

48 gb not sufficient to do multi gpu training with flux fine tuning

master_kawaii•9/17/24, 11:17 PM

I'm not ashamed I listen this...

Furkan Gözükara SECourses•9/17/24, 11:19 PM

@master_kawaii can you private message me

master_kawaii•9/17/24, 11:19 PM

Oh am I going to jail? @Dr. Furkan Gözükara haha

Furkan Gözükara SECourses•9/17/24, 11:20 PM

no something else

master_kawaii•9/17/24, 11:20 PM

oh ok xD

master_kawaii•9/17/24, 11:22 PM

There ! Master Furkan

Furkan Gözükara SECourses•9/17/24, 11:23 PM

i am testing lora extract right now out of flux fine tune

Furkan Gözükara SECourses•9/17/24, 11:25 PM

Furkan Gözükara SECourses•9/17/24, 11:25 PM

extracting right now

Furkan Gözükara SECourses•9/17/24, 11:35 PM

testing result

Furkan Gözükara SECourses•9/17/24, 11:38 PM

ye quality of lora reduced a lot

Furkan Gözükara SECourses•9/17/24, 11:39 PM

doing another test

Furkan Gözükara SECourses•9/17/24, 11:40 PM

fine tune vs lora

Furkan Gözükara SECourses•9/17/24, 11:40 PM

2336-a20photo20of20ohwx20man20wearing20an20amazing20n-Best_V1_4e_06-000150-612035258.png

2336-a20photo20of20ohwx20man20wearing20an20amazing20n-flux1-dev-612035258-1.png

Ra25•9/17/24, 11:41 PM

That's better than my lora, it has no influence even at 2.0 strength

Furkan Gözükara SECourses•9/17/24, 11:41 PM

this is str 1

Ra25•9/17/24, 11:42 PM

classic extractor have a Minimum difference you could set to 0.0001. That made all the difference when you extract.

RRa25 classic extractor have a Minimum difference you could set to 0.0001. That made a...

Furkan Gözükara SECourses•9/17/24, 11:43 PM

it was for text encoder

Furkan Gözükara SECourses•9/17/24, 11:43 PM

testing 0.01 clamp

Furkan Gözükara SECourses•9/17/24, 11:49 PM

clamp broken it

Furkan Gözükara SECourses•9/17/24, 11:55 PM

here opened an issue

Furkan Gözükara SECourses•9/17/24, 11:55 PM

https://github.com/kohya-ss/sd-scripts/issues/1610

GitHub

FLUX LoRA Extraction Resemblance Issue · Issue #1610 · kohya-ss/sd-...

I have used below config to extract 128 LoRA Network Rank with 0.99 Clamp Quantile and results are really bad What does Clamp Quantile do @kohya-ss ? Then I tried 0.01 Clamp Quantile and it is all ...

Ra25•9/18/24, 12:03 AM

Thank you for looking at the issue

Furkan Gözükara SECourses•9/18/24, 12:06 AM

ok clamp fixed

Furkan Gözükara SECourses•9/18/24, 12:07 AM

0006-a20photo20of20ohwx20man20wearing20an20amazing20n-flux1-dev-612035258.png

Furkan Gözükara SECourses•9/18/24, 12:07 AM

clamp 1 almost same

Furkan Gözükara SECourses•9/18/24, 12:09 AM

now will try 256 rank

Furkan Gözükara SECourses•9/18/24, 12:12 AM

actually extracted lora works even better haha

Furkan Gözükara SECourses•9/18/24, 12:21 AM

256 rank

Furkan Gözükara SECourses•9/18/24, 12:28 AM

ok testing 512 :d

Furkan Gözükara SECourses•9/18/24, 12:32 AM

512 rank

Furkan Gözükara SECourses•9/18/24, 12:32 AM

Furkan Gözükara SECourses•9/18/24, 12:33 AM

testing 1024

Furkan Gözükara SECourses•9/18/24, 12:38 AM

1024 failed

Furkan Gözükara SECourses•9/18/24, 12:38 AM

i will do a grid post of all and post on patreon

Ra25•9/18/24, 12:38 AM

Clamp 1 definitely helped, I also had to use a checkpoint that I considered overfitted as a finetuned model form but increased the likeness in the Lora

Furkan Gözükara SECourses•9/18/24, 12:38 AM

testing 768

has anyone got the only training specific layers yet to see a speed increase? im getting same speed

Similar Threads

Similar Threads

Similar Threads